-- Logs begin at Thu 2022-07-14 10:11:13 UTC, end at Thu 2022-07-14 11:16:49 UTC. -- Jul 14 10:42:14.663153 dv-cinder-846146-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 10:42:14 2022] *** Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: nodename: dv-cinder-846146-7-iscsi Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: machine: x86_64 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: clock source: unix Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: pcre jit disabled Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: detected number of CPU cores: 4 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: current working directory: / Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: detected binary path: /usr/bin/uwsgi-core Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: your processes number limit is 47680 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: your memory page size is 4096 bytes Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: detected max file descriptor number: 2048 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: lock engine: pthread robust mutexes Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: thunder lock: enabled Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 10:42:14.678613 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 10:42:14.719846 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: Python main interpreter initialized at 0x55b8a5bb79c0 Jul 14 10:42:14.719846 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: python threads support enabled Jul 14 10:42:14.719846 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: your server socket listen backlog is limited to 100 connections Jul 14 10:42:14.719846 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: your mercy for graceful operations on workers is 90 seconds Jul 14 10:42:14.720192 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 10:42:14.720254 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: *** Operational MODE: preforking *** Jul 14 10:42:14.720552 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: *** uWSGI is running in multiple interpreter mode *** Jul 14 10:42:14.720552 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: spawned uWSGI master process (pid: 95315) Jul 14 10:42:14.720760 dv-cinder-846146-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 10:42:14.722303 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: spawned uWSGI worker 1 (pid: 95327, cores: 1) Jul 14 10:42:14.722303 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: spawned uWSGI worker 2 (pid: 95328, cores: 1) Jul 14 10:42:14.722303 dv-cinder-846146-7-iscsi devstack@keystone.service[95315]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 10:42:20.074530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:42:20.074530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: from cryptography.utils import int_from_bytes Jul 14 10:42:20.075326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:42:20.075326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: from cryptography.utils import int_from_bytes Jul 14 10:42:20.075464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:42:20.075464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: from cryptography.utils import int_from_bytes Jul 14 10:42:20.075464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 10:42:20.075464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: from cryptography.utils import int_from_bytes Jul 14 10:42:20.123161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:42:20.123352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:42:20.123531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] command line args: [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:42:20.123768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:42:20.123998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ================================================================================ {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:42:20.124124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 10:42:20.124124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 10:42:20.124275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] admin_token = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.124341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] command line args: [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 10:42:20.124469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] config_dir = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.124596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 10:42:20.124673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.124735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ================================================================================ {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 10:42:20.124824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] config_source = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.124982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] debug = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] admin_token = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] config_dir = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] config_source = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] insecure_debug = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] debug = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.125956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] insecure_debug = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_config_append = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.126879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_dir = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_file = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_config_append = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_dir = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_file = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.127977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.128940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.129989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] notification_format = cadf {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.130967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] notification_format = cadf {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] publish_errors = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.131979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] publish_errors = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] standard_threads = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.132975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] strict_password_check = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] use_eventlog = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] use_journal = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] standard_threads = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] use_json = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] use_stderr = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] strict_password_check = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.133993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] use_syslog = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] watch_log_file = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] use_eventlog = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] use_journal = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.134589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] use_json = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.134796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] use_stderr = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.134932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] use_syslog = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.135011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] watch_log_file = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 10:42:20.135256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.135999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.136938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.137881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.138995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.139996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.140917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.141950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.142892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.143961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.144949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.145876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.146940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.147956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.148964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149600 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.149957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.150991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.151942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.enabled = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.152942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.enabled = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.153892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.154853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.155971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.156975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.157907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.158998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.external = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.mapped = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.159979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.password = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] auth.token = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.external = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.mapped = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] catalog.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160898 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.160976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.password = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] auth.token = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.161923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] catalog.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.162866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.163864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.164951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.165944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.166930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.167914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.168990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.169922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.170942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.171859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.172989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.173941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.174950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.175933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.176860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.password = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.177943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.password = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.178994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.179946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.180990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.181936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.182878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.183966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.184996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.185924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.186866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] policy.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.187969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] receipt.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] policy.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.188998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] receipt.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.189924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.190869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] revoke.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.191929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] role.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] role.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] role.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] revoke.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] role.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.192928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] role.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] role.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] role.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] role.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.193960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194876 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.194954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.195983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.196997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.197854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.198958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.199938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.200936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.provider = fernet {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.201895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.provider = fernet {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.202997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] trust.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.203971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] trust.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.204990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.connection = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.205962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.connection = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.206890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.207834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.208950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=95328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:42:20.209663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.209912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.210144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.210400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.210631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 10:42:20.210803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=95327) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 10:42:20.240543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.240741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.240940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.241032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.241223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.241442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.249375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.251990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.278867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.279087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.280819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.281040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.425596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.428503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.474041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.474179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.474264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.474338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.537926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.538994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.539063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.539124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.539291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.563751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.563938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.564913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.565034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.565165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.565245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.565363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.576040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.576167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.588914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.589975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.590054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.590128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=95327) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.590189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.590437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=95328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 10:42:20.607542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.607673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.607741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:42:20.607864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:42:20.608339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.608465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.608561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:42:20.608640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 10:42:20.608817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.608945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.609663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.620433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.620630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.620724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.620906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.620997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.621167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.621268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.621449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.629691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.630045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.633111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.633388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.633511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.633667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.633796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.633945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.634968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.635923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.636071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.661967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.662515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.663593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.663897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.664315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.664599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.669073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.669348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.669622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.669886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.670169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.670447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.676953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.677233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.677848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.678133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.681712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.681991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.682271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.682542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.682749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.682849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.682989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.683782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.684094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.684362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.684634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.702299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.702757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.703230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.703509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.707537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.708220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.711241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.711530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.711911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.712239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.716629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.717378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.720178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.721014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.723789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.724160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.724345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.724661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.724789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.724935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.725147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.725333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.725446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.725571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.725778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.726332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.744311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.744588 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.744851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.745114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.745431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.745716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.745939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.746234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.752925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.754508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.754938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.755208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.756234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.756510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.758423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.758695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.758974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.759250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.759524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.759831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.760120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.760412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.760693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.760979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.772737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.773094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.773337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.773653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.773844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.774152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.774371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.774484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.774685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.774960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.775250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.775531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.775818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.776106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.793625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.795335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.796871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.798633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.800150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.801722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.801965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.803471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.803598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.803695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.804115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.805423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.805706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.805996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.811594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.813282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.813406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.815074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.816483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.816768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.817042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.817312 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.818320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.818614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.818898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.819183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.826528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.826810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.827083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.828403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.828698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.828975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.834842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=95328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.835114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.835735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.835858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.836139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.836424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.836697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.836930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=95327) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 10:42:20.837062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.837989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=95328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.838843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.839215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.839490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.839745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.840040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.840323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.840592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=95327) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 10:42:20.876020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b8a5bb79c0 pid: 95328 (default app) Jul 14 10:42:20.878257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b8a5bb79c0 pid: 95327 (default app) Jul 14 10:42:20.879774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22028-2613-4825-8899-65984c539134 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:20.880089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22028-2613-4825-8899-65984c539134 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:20.880363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22028-2613-4825-8899-65984c539134 None None] PATH_INFO: `/v3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:20.884646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Thu Jul 14 10:42:20 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:28.565274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:28.566188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:28.566188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:28.572189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:28 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:28.578146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:28.578586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:28.578676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:28.647764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: INFO dbcounter [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] Registered counter for database keystone Jul 14 10:42:28.672688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG oslo_db.sqlalchemy.engines [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=95328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:42:28.684534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writer thread running {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:42:28.865777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG passlib.handlers.bcrypt [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=95328) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:42:28.866180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG passlib.handlers.bcrypt [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=95328) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:42:28.924019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:28.925666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:29.060705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 10:42:29.384388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:29.465760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: INFO dbcounter [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] Registered counter for database keystone Jul 14 10:42:29.487313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG oslo_db.sqlalchemy.engines [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=95327) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 10:42:29.498788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writer thread running {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 10:42:29.620120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5b698-590c-4c50-9e7f-7c82a5b033c6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:29.628592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:29.630011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:29.630491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:29.630948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:29.646045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:29.698627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:42:29.698973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f054cf6-4df5-4ac7-8f95-6f3de266b784 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:29.709715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Thu Jul 14 10:42:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:42:29.716377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:29.731988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9809a-3207-4894-b8f7-f76d72528bc9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:29.737639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:29.738774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:29.739063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:29.739336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] PATH_INFO: `/v3/projects/dd0314351a554271b0b9af50b68ac028` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:29.743710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] RBAC: Authorizing `identity:get_project(project_id=dd0314351a554271b0b9af50b68ac028)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:29.794671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 10:42:29.795008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb564a6-b89a-43b2-95a0-31415ddae345 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:29.799693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Thu Jul 14 10:42:29 2022] GET /identity/v3/projects/dd0314351a554271b0b9af50b68ac028 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:31.783098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a16fd-0269-47dc-b319-cf2486aac640 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:31.783673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a16fd-0269-47dc-b319-cf2486aac640 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:31.783673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a16fd-0269-47dc-b319-cf2486aac640 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:31.787863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:42:31.794696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:31.795401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:31.795800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:31.820056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:31.821820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:31.831071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 10:42:31.854058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a16fd-0269-47dc-b319-cf2486aac640 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:31.867342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a16fd-0269-47dc-b319-cf2486aac640 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:31.872722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:31.873359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:31.873644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:31.873920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:31.882960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:31.884786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174cd4e-3e41-4659-9510-f5ddb64a9f63 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:31.894048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Thu Jul 14 10:42:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:42:31.899902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:31.910109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23a39-e79a-4b5d-861c-4cb9d42358af None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:31.916957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:31.918273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:31.918667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:31.919051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:31.921509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:31.924540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348a217-f043-438c-9626-56c2a76351c9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:31.927076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Thu Jul 14 10:42:31 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:34.046636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2b35e-8234-472c-909e-e6a52ad64047 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.047169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2b35e-8234-472c-909e-e6a52ad64047 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.047249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2b35e-8234-472c-909e-e6a52ad64047 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.048444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d3b92-3379-480b-aa3a-7aeba65d9e53 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.048824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d3b92-3379-480b-aa3a-7aeba65d9e53 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.049081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d3b92-3379-480b-aa3a-7aeba65d9e53 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.049168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:34.051090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:34.054880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.055411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.055704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.056409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.056701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.056981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.074047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:34.075250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:34.096072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG passlib.handlers.bcrypt [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=95327) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 10:42:34.096491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG passlib.handlers.bcrypt [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=95327) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 10:42:34.141970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:34.143578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:34.162311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 7/12] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:34.185309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.211444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.246612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 6/13] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:34.268897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.295400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6246-9d91-48c4-a391-fbd55b9fadea None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.328512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.329314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.329597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.329870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.341285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:34.352546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:34.353459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:34.354263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 8/14] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:34.360025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.369704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fda9e-2d33-4555-8355-3c1290f6f2db None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.372730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.410772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.411811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.412132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.412413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.441889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.460288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0feb4-79ae-47a1-b211-79eaefb73ce1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.482867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.484634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.495072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.495742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.496058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.496337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.497130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:34.497438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.498872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.511647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 9/15] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:34.517441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.592084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 7/16] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:42:34.597721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.600481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.615492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c51e-5500-49f1-ae8f-06147771afcf None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.650687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.651565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.651853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.652161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.653415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.654912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.656601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 10/17] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:34.661978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.676814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.698776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5846-2da3-4649-b94d-9ae96e3b4773 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.732405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.733260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.733479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.733754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.753384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.755231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.760574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce8192-b99d-447e-bbfa-f1af99d1155c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:42:34.776286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:34.777243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:34.778055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:42:34.778055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:42:34.778055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:42:34.778055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:34.778055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 8/18] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:42:34.783638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.794260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.795070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.795343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.795616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.797090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.798564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.800242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 11/19] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:34.805989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:34.841479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.885618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:34.886620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47f566-bb7a-41c0-8c44-2a8b635a04b9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.905358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769ae3b-48cd-418f-9df3-52ccbda664a6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:34.919942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.920622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.920904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.921185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.921919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:34.922227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.923646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dfa46-0354-4687-8505-6394e59103b1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.940958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 9/20] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:34.941750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:34.942420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:34.942705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:34.942986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:34.943834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.domain.id=default, user.id=e56c84bcaa9a49c8bd5f12e10aeab889) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:34.944184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:34.945943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bc6a8-e011-4444-abbd-c15fd3a71403 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:34.953679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 12/21] 10.222.0.27 () {56 vars in 1322 bytes} [Thu Jul 14 10:42:34 2022] GET /identity/v3/role_assignments?user.id=e56c84bcaa9a49c8bd5f12e10aeab889&scope.domain.id=default&role.id=bd93d91744f54f739650facbdec5aef5 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:37.271103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312ee2-a9a4-46dc-a102-3f2c3a5a9387 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.272003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312ee2-a9a4-46dc-a102-3f2c3a5a9387 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.272498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312ee2-a9a4-46dc-a102-3f2c3a5a9387 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.274567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 10/22] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:37.280541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.281499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.281978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.308862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:37.310391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:37.397449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 13/23] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:37.421263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c312ee2-a9a4-46dc-a102-3f2c3a5a9387 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:37.530748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c312ee2-a9a4-46dc-a102-3f2c3a5a9387 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:37.548580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:37.565404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:37.566214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.566493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.566768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.579632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-1cd31ceb-da5f-48b2-aa95-9398b6405a46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:37.588834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:37.589761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:37.590550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 11/24] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:37.596441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:37.676496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:37.702101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b613f0-af97-410f-bf72-4604323b035a None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.702428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b613f0-af97-410f-bf72-4604323b035a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.702656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b613f0-af97-410f-bf72-4604323b035a None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.704364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 12/25] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:37.706071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfb303-5970-4f58-ac9e-f3cf2a2f72e3 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:37.709132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9f84-3e26-4ea9-a238-05bd4f8da821 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.709413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9f84-3e26-4ea9-a238-05bd4f8da821 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.709688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9f84-3e26-4ea9-a238-05bd4f8da821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.740203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-049d9f84-3e26-4ea9-a238-05bd4f8da821 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:37.741393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-049d9f84-3e26-4ea9-a238-05bd4f8da821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:37.773353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:37.774098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.774379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.774650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.775408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:37.776028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:37.777599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:37.787514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 14/26] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:37.793541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:37.914130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 13/27] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:37.916157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b9c4-eb69-4526-9e20-1ea0d820afef None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.916360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b9c4-eb69-4526-9e20-1ea0d820afef None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.916612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b9c4-eb69-4526-9e20-1ea0d820afef None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.918124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 14/28] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:37.929258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:37.929586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:37.930742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:37.942401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e19e5-30ae-4739-90d9-abd35b475f4b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:37.965807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:37.967015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:38.000127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.001018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.001300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.005527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.006816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.008713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.010595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 15/29] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:37 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:38.017524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.032255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.087896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 15/30] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:38.090284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.146448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.163586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6cd19-80d5-4c7c-b841-a74279e55835 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.202409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.203504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.204079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.204527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.216905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.218704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.230048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d04d-0463-4a26-860a-de66d1882e0c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-93674acb-a6d8-47f0-aa8c-cf1fccbde9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:42:38.239484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:38.240508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:38.241466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:42:38.241466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:42:38.241466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:42:38.241466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:38.243708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:38.249259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233e12-57e1-49de-8904-1755173b61ef None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.249718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233e12-57e1-49de-8904-1755173b61ef None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.250199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233e12-57e1-49de-8904-1755173b61ef None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.251624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 17/32] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:38.253961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233e12-57e1-49de-8904-1755173b61ef None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.275102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.276101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.276282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.276507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.278232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.280273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.282531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 16/33] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:38.283670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.327511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.385160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233e12-57e1-49de-8904-1755173b61ef None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.413449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377a4c1-90a5-4e3c-94b6-b431fed74275 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.416221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.428115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.429134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.429453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.429952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.431132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.432652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.454619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.455295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.455575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.455901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.456608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:38.456914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.458291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658244a7-6f8a-465e-86dd-053e0a3d6872 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.475203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 17/34] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:38.475203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5137d-83b0-4322-a1eb-00c4ccc508bf None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.475203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5137d-83b0-4322-a1eb-00c4ccc508bf None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.475203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5137d-83b0-4322-a1eb-00c4ccc508bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.490644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d1c5137d-83b0-4322-a1eb-00c4ccc508bf None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:38.491804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d1c5137d-83b0-4322-a1eb-00c4ccc508bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:38.563918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 18/35] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/roles => generated 225 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:38.563918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.573021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 18/36] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:38.574651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7822c-c705-440c-b53d-c0c3c213a106 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.574957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7822c-c705-440c-b53d-c0c3c213a106 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.575231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7822c-c705-440c-b53d-c0c3c213a106 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.581168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 19/37] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:38.582192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7822c-c705-440c-b53d-c0c3c213a106 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.702513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e94fc-8518-4edb-84ad-bff1228e4c24 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.717280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.720693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:38.731032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7822c-c705-440c-b53d-c0c3c213a106 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:38.741070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.742093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.742431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.742711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] PATH_INFO: `/v3/domains/default/users/e56c84bcaa9a49c8bd5f12e10aeab889/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.745364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e56c84bcaa9a49c8bd5f12e10aeab889, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.746991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adc94f-e681-44a9-bfaf-ca9925433b05 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.769413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 19/38] 10.222.0.27 () {58 vars in 1361 bytes} [Thu Jul 14 10:42:38 2022] PUT /identity/v3/domains/default/users/e56c84bcaa9a49c8bd5f12e10aeab889/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:38.771129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.771620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.772134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.794056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:38.795260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:38.796051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:38.796378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:38.796658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:38.797044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:38.797491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:38.799108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:38.887789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 20/39] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:38.888782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:38.980862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 20/40] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:38.982259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:39.005497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e7c78-149f-4c59-86e6-f3cf75c01208 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:39.014982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:39.049891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:39.050583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.050871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.051153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.051970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:39.053415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f347d6-6b89-467a-be06-b6c8e804e7b4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:39.074260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 21/41] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/roles => generated 219 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:39.080555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e672-3b18-4a86-aaef-b95cef4b5626 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.081196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e672-3b18-4a86-aaef-b95cef4b5626 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.081685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e672-3b18-4a86-aaef-b95cef4b5626 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.083755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 22/42] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:39.089366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.089988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.090453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.119423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:39.121108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:39.138104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f2d92-834a-4ac3-aa26-0566a7eb798d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:39.187073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:39.187787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.188109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.188431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.189159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:39.190733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:39.210472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 21/43] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Jul 14 10:42:38 2022] POST /identity/v3/roles => generated 223 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:39.380568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 23/44] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:42:39.404067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:39.414516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:39.515225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c670f-0ba4-4e25-b1fc-55815d57aeba None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:39.548035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:39.548835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.549119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.549395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.550659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:39.552163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111036e-72ff-4c25-ac4e-c572b82896c6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:39.553832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 22/45] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:39.559828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:39.659739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:39.666619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f372b6-6528-4a80-b1af-c7f0c952ed6a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:39.700544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:39.701200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:39.701481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:39.701752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:39.702460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:39.703976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a952-b89f-4732-b5a0-14985bedad99 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:39.731473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:42:39 2022] POST /identity/v3/projects => generated 312 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:40.932346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07ec2-8da7-4814-bc0e-59407ee83032 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:40.932915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07ec2-8da7-4814-bc0e-59407ee83032 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:40.932915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f07ec2-8da7-4814-bc0e-59407ee83032 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:40.934501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:40.939943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:40.939943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:40.940377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:40.957719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:40.958893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:41.027773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:41.050723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f07ec2-8da7-4814-bc0e-59407ee83032 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.128682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:41.158427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f07ec2-8da7-4814-bc0e-59407ee83032 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:41.193348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:41.194083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.194369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.194648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.204883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:41.212559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:41.213456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:41.214203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:41.219901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.301290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:41.325016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2e67-bcf0-4621-ab1c-a5b2032b04a1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:41.359041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:41.359716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.360023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.360265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.361054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:41.361366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:41.362782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:41.371681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 26/50] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:41.377278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.484153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0331a4-9c70-4759-818c-563593a66588 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:41.520602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:41.521504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.521767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.522049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.523649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:41.525228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:41.526925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 25/51] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:41.532751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.634700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5211e7-592d-4978-8c01-5a5d79c5f129 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:41.671055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:41.672281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.672701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.673118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.674863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:41.677022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244b9fa-e868-4f46-8ff2-f93c62a791b7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:41.679635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 27/52] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:41.685804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.689142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:41.747599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655352-5d41-43db-b215-5146320c8c2a None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.747936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655352-5d41-43db-b215-5146320c8c2a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.748230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea655352-5d41-43db-b215-5146320c8c2a None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.749687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 28/53] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:41.754755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.755388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.755676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.772948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:41.774263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:41.776446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:41.796410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f0f3c-fb55-49e5-8ac4-be7b3d51c8d9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:41.833860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:41.834521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:41.834803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:41.835081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:41.836034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.domain.id=default, user.id=e56c84bcaa9a49c8bd5f12e10aeab889) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:41.836432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:41.838027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:41.843651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 29/54] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:41.846827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 26/55] 10.222.0.27 () {56 vars in 1322 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/role_assignments?user.id=e56c84bcaa9a49c8bd5f12e10aeab889&scope.domain.id=default&role.id=bd93d91744f54f739650facbdec5aef5 => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:41.868527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:41.984356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c49fdf-6a0c-4d3e-8c5b-2cd87c64d601 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:42.025920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:42.026634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:42.026913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:42.027192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:42.028434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:42.029930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e42ef1-5d22-45d9-9991-f1934b2e111e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:42.031564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 30/56] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:42.038105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:42.106594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:42.254505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c171-941d-4d4c-9e48-00b77e19a8e7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:42.292232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:42.292819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:42.293108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:42.293386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:42.294118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:42.295596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304dcde-ca04-4e73-8715-f96a472f603f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:42.324519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 27/57] 10.222.0.27 () {60 vars in 1140 bytes} [Thu Jul 14 10:42:42 2022] POST /identity/v3/projects => generated 298 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:42:44.333868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a84f-7c79-4fc2-ba06-ccaf2a1ad176 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:44.334405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a84f-7c79-4fc2-ba06-ccaf2a1ad176 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:44.334405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945a84f-7c79-4fc2-ba06-ccaf2a1ad176 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:44.335720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 31/58] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:44.342051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:44.342477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:44.342692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:44.365644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:44.367367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:44.443218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 28/59] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:44.469108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945a84f-7c79-4fc2-ba06-ccaf2a1ad176 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:44.481006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:44.583684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945a84f-7c79-4fc2-ba06-ccaf2a1ad176 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:44.618683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:44.619510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:44.619779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:44.620071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:44.621310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:44.622869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228038e8-e41c-44e9-9f9d-c300673b2244 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:44.624506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 32/60] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:44.632843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:44.682258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:44.732944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52b457-5720-4eef-b465-f3822b7ace60 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:44.768240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:44.768903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:44.769181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:44.769457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:44.770173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:44.771635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a217c2f-4fbe-497d-bdeb-c0963c006187 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:44.802131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 29/61] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:42:44 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:47.910619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3771ab-78ec-44b8-8190-9e0169eafafc None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:47.911486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3771ab-78ec-44b8-8190-9e0169eafafc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:47.911927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3771ab-78ec-44b8-8190-9e0169eafafc None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:47.913769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 33/62] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:47.918877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c0ebb-8baa-46e8-ac5c-21775d1ed5e9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:47.919662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c0ebb-8baa-46e8-ac5c-21775d1ed5e9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:47.920132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c0ebb-8baa-46e8-ac5c-21775d1ed5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:47.949002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-357c0ebb-8baa-46e8-ac5c-21775d1ed5e9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:47.950222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-357c0ebb-8baa-46e8-ac5c-21775d1ed5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:48.049275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 30/63] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:48.082798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3771ab-78ec-44b8-8190-9e0169eafafc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.110144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f18-8990-4c9f-a02b-fb7468b90074 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.110488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f18-8990-4c9f-a02b-fb7468b90074 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.110707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c68f18-8990-4c9f-a02b-fb7468b90074 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.112147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 31/64] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:48.118470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f7f0-70b7-42c5-bd51-0c044705c1cf None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.118757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f7f0-70b7-42c5-bd51-0c044705c1cf None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.119038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56f7f0-70b7-42c5-bd51-0c044705c1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.137147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ca56f7f0-70b7-42c5-bd51-0c044705c1cf None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:48.138770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ca56f7f0-70b7-42c5-bd51-0c044705c1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:48.168224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.221708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.224876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 32/65] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:48.226679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dbca6-95ae-44a1-95ce-20ab29c87a90 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.227373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dbca6-95ae-44a1-95ce-20ab29c87a90 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.228266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68dbca6-95ae-44a1-95ce-20ab29c87a90 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.230485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 33/66] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:48.237847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.238401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.238891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.256755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:48.258467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:48.278320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3771ab-78ec-44b8-8190-9e0169eafafc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.325197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.326375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.326872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.327481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.338990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 34/67] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:42:48.344082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.344748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:48.357373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:48.358248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:48.359018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:48.361628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 34/68] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:48.368059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.438596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.446270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de882-c203-4aca-8e48-6a4f841368e1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.483609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.484687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.484976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.485255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.495604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:48.498691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a64d-9845-4271-a044-5a36489d7d7e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:48.504043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:48.505089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:48.506328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:48.508383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 35/69] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:48.509438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.539681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.540363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.540646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.540919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.541669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:48.541978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:48.543391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0aae2-d932-4e4d-958f-1b14a7aff611 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:48.544910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.556184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 35/70] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:48.558400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bde3c-f835-48c4-8653-f341746e0dd0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.558691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bde3c-f835-48c4-8653-f341746e0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.558972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bde3c-f835-48c4-8653-f341746e0dd0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.560430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 36/71] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:48.561068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bde3c-f835-48c4-8653-f341746e0dd0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.621199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ce566-aeb5-485f-8a95-2949484cee3a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.658832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.659709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.660025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.660328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.670889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:48.674529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bde3c-f835-48c4-8653-f341746e0dd0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:48.679001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:48.680027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:48.680813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 36/72] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:48.681437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.702382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.710744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.711450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.711737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.712050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.712811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:48.713123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:48.714551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d854622-fe84-486f-8a8e-bfc866397753 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:48.722620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 37/73] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:48.723891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958a410-7931-4f45-abc3-fafe118eb756 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.724199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958a410-7931-4f45-abc3-fafe118eb756 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.724482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958a410-7931-4f45-abc3-fafe118eb756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.740583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5958a410-7931-4f45-abc3-fafe118eb756 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:48.742066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5958a410-7931-4f45-abc3-fafe118eb756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:48.788731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2bf0a-c27f-48b1-8af9-7d0ecf7b3061 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.797762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.811608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 38/74] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:48.813132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8cf48-64a5-4258-ab9a-b0ed10f49af5 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.813421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8cf48-64a5-4258-ab9a-b0ed10f49af5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.813698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8cf48-64a5-4258-ab9a-b0ed10f49af5 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.814975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 39/75] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:48.815664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8cf48-64a5-4258-ab9a-b0ed10f49af5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.831586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.832499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.832786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.833068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.848924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:48.850454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:48.852362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 37/76] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:48.853137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:48.923358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:48.931293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8cf48-64a5-4258-ab9a-b0ed10f49af5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.957577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453a87c-3d2d-401d-b5fb-8ea8598a12b9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:48.968266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.968945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.969238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.969517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.970277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:48.970589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:48.972050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840d068-8eda-410a-ba19-b2d05a6950e5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:48.980580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 40/77] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:48.981839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.982136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.982421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.993673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:48.994542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:48.994825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:48.995098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:48.996369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:48.997851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:48.998506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:48.999506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 38/78] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:48.999697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:49.000286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.046397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.069607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 41/79] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:49.070470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.104567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dbd70-d67e-466c-8d35-ce481de97ca6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.144069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.145099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.145420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.145753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.154784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.158103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:49.165976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:49.166850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:49.167639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 39/80] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:49.168310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.182087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60faf2-8d9f-462b-a0df-35846b3caed5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.217410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.217983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.218264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.218534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.266217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2751-df9f-484b-b9ec-52f00634769f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.266690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.286384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.287911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.300986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.301873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.302164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.302449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.303735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.305599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.307431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 40/81] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:49.308072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.390650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.416239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 42/82] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:49.420901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.432475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fb62b-b68c-461e-9651-1164a8372bf0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.467531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.468148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.468440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.468723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.509125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.533363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114037b-ec8e-4444-a0dc-eafef4bfb54c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.537767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.539365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.568301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.569118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.569405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.569686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.570929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.572465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.578634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 43/83] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:49.578634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.616143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.651013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 41/84] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:49.651866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.685967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d9751-1238-4664-a71b-804bd1c73bd6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.718974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.719567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.719854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.720190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.720954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:49.721260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.722694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.730737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 44/85] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:49.731525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.743529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.750186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc2cff-609a-4526-8b62-90100490487f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.785952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.786603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.786885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.787164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.821259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.828218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3608e-f9b8-4209-baad-cdad6b779c35 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:49.855188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.856745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.860426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:49.861299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:49.861582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:49.861859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:49.863037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:49.864533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:49.866270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 45/86] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:49.867001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.946395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:49.959346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 42/87] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:49.960398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:49.968544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85997e2c-2568-47d7-b598-9f874691cc27 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.001184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.001789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.002072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.002351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.003072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.004568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.027802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.030710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 46/88] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:42:49 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:50.031530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.081361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931f69-23aa-4ad2-9c67-bda78a8f7b33 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.120064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.120970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.121256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.121520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.122796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.124345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.126144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 43/89] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:49 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.126952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.150791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea153e2-c84b-4d70-9f6b-e097978c6564 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.189873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.190783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.191173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.191351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.192800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.194697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.196553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 47/90] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.201142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.201142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.247240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55286b87-ce96-47bb-bd38-da81b7ccbc23 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.287850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.288846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.289511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.289963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.290997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:50.291487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.293168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.297902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.301412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 44/91] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=8cde109646d74ea789591d2724109ef7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.302453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.331577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de095d1d-4432-46cf-9df1-a3e89886130b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.386094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.387320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.387776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.388262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.389652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.391340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.393434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 48/92] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.394174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.465786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ea784-74b5-4a7d-bb37-e70e595b85a4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.489715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.513936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.514613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.514899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.515177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.516500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=e56c84bcaa9a49c8bd5f12e10aeab889) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:50.516823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.518742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.526128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 45/93] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/role_assignments?user.id=e56c84bcaa9a49c8bd5f12e10aeab889&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.527364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:50.529554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73722db-7edd-4d3e-b0f3-cfb17095301d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.589257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.589904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.590193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.590467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.720796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.722398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.744993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.835279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5a3d-0773-4f3a-8f94-a14b8d3c2b03 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:50.856830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:50.916863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:50.917630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:50.917950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:50.918266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:50.919085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:50.919396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:50.920908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:50.923829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 49/94] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:50.928590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 46/95] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 259 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:50.933182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:51.067253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2fc34-44a5-4bbe-a0fc-9937fc70ab74 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:51.103334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:51.104249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:51.104532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:51.104804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] PATH_INFO: `/v3/projects/030c170c7f324776ab88181f1fc096d9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:51.105996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] RBAC: Authorizing `identity:get_project(project_id=030c170c7f324776ab88181f1fc096d9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:51.107513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f33be-6f76-4a91-87c1-2d2ea88631c1 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:51.112043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 50/96] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:50 2022] GET /identity/v3/projects/030c170c7f324776ab88181f1fc096d9 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:51.115252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:51.261384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bb9c7-2c4e-4772-b626-b0120c54f93b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:51.299155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:51.312127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:51.312807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:51.313090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:51.313369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:51.314218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=030c170c7f324776ab88181f1fc096d9, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:51.314533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:51.316036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8ee38-6d72-42dc-bac5-3e4edabead56 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:51.322601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 47/97] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:51 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=030c170c7f324776ab88181f1fc096d9&role.id=8cde109646d74ea789591d2724109ef7 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:52.840896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627297-5f1c-4847-bab3-0501164433f6 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:52.841460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627297-5f1c-4847-bab3-0501164433f6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:52.841460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62627297-5f1c-4847-bab3-0501164433f6 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:52.843079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 51/98] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:52.848428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb26ba-1783-442a-b6ed-7c833ff8d32b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:52.848823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb26ba-1783-442a-b6ed-7c833ff8d32b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:52.848998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb26ba-1783-442a-b6ed-7c833ff8d32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:52.865943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c0fb26ba-1783-442a-b6ed-7c833ff8d32b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:52.867119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c0fb26ba-1783-442a-b6ed-7c833ff8d32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:52.957430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 48/99] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:52.992807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62627297-5f1c-4847-bab3-0501164433f6 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.044975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.113496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62627297-5f1c-4847-bab3-0501164433f6 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.150630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.151443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.151726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.152034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.156024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c94a-b8a8-4896-8e81-260d4d64a80e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.156315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c94a-b8a8-4896-8e81-260d4d64a80e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.156581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c94a-b8a8-4896-8e81-260d4d64a80e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.158014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 49/100] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:42:53.164256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.164540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.164815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.183068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:53.191072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:53.191964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:42:53.192739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:53.196149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:53.197482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:53.199659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 52/101] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:53.206578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.299394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 50/102] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:42:53.322525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.331643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.337301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287da33-6285-48f5-9542-edbd612f91cd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.383560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.385794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.387038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.387762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.388080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.389245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:53.389566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:53.392925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:53.405472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 53/103] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:53.412630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.435080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56c88-256b-4395-baed-8e16e99c4543 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.477508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.478336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.478626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.478909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.493833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-824f2f3f-d837-4936-8ff5-c496a8306e23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:53.504396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:53.505232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:53.506349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:53.511897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 51/104] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:53.511897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7903-2f0b-4f2d-a751-9cc39767312f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.511897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7903-2f0b-4f2d-a751-9cc39767312f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.511897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a7903-2f0b-4f2d-a751-9cc39767312f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.511897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 52/105] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:53.517088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a7903-2f0b-4f2d-a751-9cc39767312f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.541396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eb205-238a-4672-8936-e27a8edeca14 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.625471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.626879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.627308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.627766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] PATH_INFO: `/v3/users/dc78cacc045349fabeddccb7d8c6557f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.638956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a7903-2f0b-4f2d-a751-9cc39767312f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.662591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.666261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] RBAC: Authorizing `identity:get_user(user_id=dc78cacc045349fabeddccb7d8c6557f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:53.667787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3b7f-73e5-4e11-9b91-f85466a31b79 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:53.669602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 54/106] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/users/dc78cacc045349fabeddccb7d8c6557f => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:53.678536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cf602-37a9-48d2-b70c-71813f8fb7fd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.678832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cf602-37a9-48d2-b70c-71813f8fb7fd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.679116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cf602-37a9-48d2-b70c-71813f8fb7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.701065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-200cf602-37a9-48d2-b70c-71813f8fb7fd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:53.702273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-200cf602-37a9-48d2-b70c-71813f8fb7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:53.711060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.711864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.712202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.712528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.713664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:53.713979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:53.716043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:53.716978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.730958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 53/107] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:53.731807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.794424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 55/108] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:53.796474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2775b7-3bf6-4af5-8483-4be451c5be17 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.796774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2775b7-3bf6-4af5-8483-4be451c5be17 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.797112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2775b7-3bf6-4af5-8483-4be451c5be17 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.798475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 56/109] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:53.799288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2775b7-3bf6-4af5-8483-4be451c5be17 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:53.870757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c196f-4a0a-4bcc-9697-69221be818ba None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.911982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.913078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.913872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.914604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.941501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2775b7-3bf6-4af5-8483-4be451c5be17 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:53.969895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:53.990039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:53.991985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:53.992504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:53.993855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:53.995577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:53.997665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.000304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.005123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc39b0-133d-42de-b8f9-927dd8c0158f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.005242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 57/110] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.005242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.024453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.106206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79dc29-8b58-4163-b410-b582e7b98d1a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.117316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 54/111] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:54.119090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.119585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.120368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.140611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:54.142343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:54.146437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.147259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.147529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.147801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.158466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:54.166498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:54.167392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:54.168586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 58/112] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:54.173075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.222116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 55/113] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:54.223111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.226113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.257184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.282480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f494-d834-4c4c-a6ec-0b54f15af128 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.326684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.327577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.327860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.328175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.329493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.329730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97401c75-5c35-4cf3-9b87-4ffa812d3647 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.331008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.332928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 59/114] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.333641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.371918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.372799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.373085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.373361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.374561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.376089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a50cad-ba9e-448c-97f5-a9e25f9b26e7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 56/115] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ad78-30c7-479d-819a-2a4c82d2dd24 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ad78-30c7-479d-819a-2a4c82d2dd24 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1ad78-30c7-479d-819a-2a4c82d2dd24 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 57/116] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:54.382841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1ad78-30c7-479d-819a-2a4c82d2dd24 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.445558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd27f0-8ce7-4077-a0c5-4030a9f98cd2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.463513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.484078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.485122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.485590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.486004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.487103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:54.487488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.489814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.508213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1ad78-30c7-479d-819a-2a4c82d2dd24 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.514133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 60/117] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.519118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.528743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.547757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.548602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.548862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.549144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.550364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.551840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.558311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 58/118] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.558311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.638601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86f04e-68c4-4e4e-966e-42a2ccdf17ab None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.666065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6d8b-8f78-49d6-878f-35458592c71b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.675487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.676431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.676725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.676997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/36056ea955f24614b41b93e8556945e6/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.679030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, user_id=36056ea955f24614b41b93e8556945e6, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.680798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896185f-3cb4-443c-850c-5a864efd66de None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.695095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 61/119] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:42:54 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/36056ea955f24614b41b93e8556945e6/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:54.697202 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.697422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.697706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.707380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.708471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.708907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.709330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.710710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=dc78cacc045349fabeddccb7d8c6557f) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:54.711440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.713831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841c437-7452-4397-93e5-d16017ab7dbc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.720913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:54.722554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:54.723237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 59/120] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/role_assignments?user.id=dc78cacc045349fabeddccb7d8c6557f&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.725549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478ee0-5b80-4fe7-840a-af2177a1ad70 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.725980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478ee0-5b80-4fe7-840a-af2177a1ad70 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.726303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68478ee0-5b80-4fe7-840a-af2177a1ad70 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.728334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 60/121] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:54.729193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68478ee0-5b80-4fe7-840a-af2177a1ad70 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.742405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.844799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:54.858012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 62/122] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:54.859522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:54.897279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68478ee0-5b80-4fe7-840a-af2177a1ad70 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:54.957788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:54.959711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.960935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.961768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.963257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:54.965611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc756-8e2f-4528-bed9-7d8f33f00bc2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:54.967752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 61/123] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:54 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:54.970164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:54.970658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:54.971086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:54.991308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:54.993843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:55.001631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5d01b-4b4b-4c76-80b7-65f018c47edf None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.043815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.044798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.045413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.046012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.047129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:55.048773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:55.088724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 62/124] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:55.090707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.096194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 63/125] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:42:54 2022] POST /identity/v3/projects => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:55.097678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:55.097791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.181332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:55.277909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e54285-a05c-4c51-a810-cc476a4a5c83 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.304481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8e5f0-7d00-437f-8d99-31ad316a75fa None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.337919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.339967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.340462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.341200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.342777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:55.344791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:55.347818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 64/126] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:55.348893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.371242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.371242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.371242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.371579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.385285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:55.394666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:55.396465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 63/127] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:55.397517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.485282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5fe1d-f46c-4407-9caf-160d6d1a7aa7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.514926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:55.526231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:55.548792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.550634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.551371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.552130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.574457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:55.595193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:55.596761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:42:55.597969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:55.601582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 65/128] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:55.603516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.685891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786c621-c112-4d6b-9061-25836bcdf106 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.790745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304b65c-fbc5-4fe3-8a28-e5c7bc697196 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:55.801579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.802425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.802714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.802995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/e56c84bcaa9a49c8bd5f12e10aeab889/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.805090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, user_id=e56c84bcaa9a49c8bd5f12e10aeab889, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:55.806590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:55.841264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 64/129] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:42:55 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/e56c84bcaa9a49c8bd5f12e10aeab889/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:55.841870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.855559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:55.857252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:55.858226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:55.859092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:55.860777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:55.861640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:55.865276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:55.887832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 66/130] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:55.893748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:55.972765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:56.084989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc543a02-4f5d-4e27-b162-a8250bb525d6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.142405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb6cda-fbbd-4216-a284-53579e5f653d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.157007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.158905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.159628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.164432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.167116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.170236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.173736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 67/131] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:56.182494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:56.202919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:56.225744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.226441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.226788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.227052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.228277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:56.232360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.233791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.250472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 65/132] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:42:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:56.259344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:56.372694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74ed09-32a9-45da-8015-29bf16cda818 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.393703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:56.479549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.480853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.481455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.482723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] PATH_INFO: `/v3/projects/030c170c7f324776ab88181f1fc096d9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.486745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] RBAC: Authorizing `identity:get_project(project_id=030c170c7f324776ab88181f1fc096d9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.488701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.491733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 68/133] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:56 2022] GET /identity/v3/projects/030c170c7f324776ab88181f1fc096d9 => generated 312 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:56.498158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:56.522593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3098707-2cf2-4eb8-b77a-2521fc92daa6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.615580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.616653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.617001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.617366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.619435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.625615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.627820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 66/134] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:56 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:42:56.635900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:56.728878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388403d1-8788-4159-8116-ebd7492b3866 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.736318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:56.752439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212be78b-79ff-4b4c-9dc7-61d558d62c6f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:56.798528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.799452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.799738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.800048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] PATH_INFO: `/v3/projects/030c170c7f324776ab88181f1fc096d9/users/36056ea955f24614b41b93e8556945e6/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.805409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] RBAC: Authorizing `identity:create_grant(project_id=030c170c7f324776ab88181f1fc096d9, user_id=36056ea955f24614b41b93e8556945e6, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.807013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37065d8f-8696-462a-9250-9862c38ad774 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.824403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:56.825296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:56.825584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:56.825858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:56.830032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:56.832134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:56.833900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 67/135] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:56 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:56.834456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 69/136] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:42:56 2022] PUT /identity/v3/projects/030c170c7f324776ab88181f1fc096d9/users/36056ea955f24614b41b93e8556945e6/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:56.845485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:56.989555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e695-3cf5-4ee9-a224-36d552a9fcf8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:57.014627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:57.036961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:57.037875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:57.038158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:57.038432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/36056ea955f24614b41b93e8556945e6/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:57.041890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, user_id=36056ea955f24614b41b93e8556945e6, role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:57.043424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d6725-016c-48d0-bffa-29f2f488db4c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:57.073761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 68/137] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:42:56 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/36056ea955f24614b41b93e8556945e6/roles/a0d79cab834140ccb511ceb372f735a2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:57.419619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762ea5b-3841-42c7-80be-a28421af4eca None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:57.420194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762ea5b-3841-42c7-80be-a28421af4eca None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:57.420194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762ea5b-3841-42c7-80be-a28421af4eca None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:57.421626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 70/138] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:57.427834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:57.428293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:57.428477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:57.453353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:57.454577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:57.549617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 69/139] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:57.591714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762ea5b-3841-42c7-80be-a28421af4eca None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:57.638779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:57.728673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762ea5b-3841-42c7-80be-a28421af4eca None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:57.772603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:57.773500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:57.773962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:57.774286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:57.788973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:57.796882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:57.797758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:57.798556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 71/140] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:57.804143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:57.946231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47765cd-75d6-43aa-9345-ce6a8e5d0b17 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:57.968215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.003593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.004223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.004512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.004794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.005561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:58.006214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:58.007639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f818-c021-4605-81d4-8b7d6147d8b3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:58.023285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 70/141] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:58.028766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.074545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103242-b13a-46e7-a74b-ef528cd80886 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.074882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103242-b13a-46e7-a74b-ef528cd80886 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.075118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74103242-b13a-46e7-a74b-ef528cd80886 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.076547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 71/142] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:42:58.082711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbe9c-69cf-4826-a3ac-b77c1abe4a08 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.083001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbe9c-69cf-4826-a3ac-b77c1abe4a08 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.083287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbe9c-69cf-4826-a3ac-b77c1abe4a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.111664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3f0dbe9c-69cf-4826-a3ac-b77c1abe4a08 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:58.114902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3f0dbe9c-69cf-4826-a3ac-b77c1abe4a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:58.181393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.199441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de01898-089d-4b9f-9da1-208822d97555 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:58.238104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 72/143] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:42:58.239512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1860a43-b1d0-4bf9-8ef1-9aa01f776fe2 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.239804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1860a43-b1d0-4bf9-8ef1-9aa01f776fe2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.240111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1860a43-b1d0-4bf9-8ef1-9aa01f776fe2 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.241620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 73/144] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:58.249337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.249629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.249912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.271675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.272520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.272798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.273071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.276052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:58.277545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:58.279435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 72/145] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:58.280227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.284183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:58.285365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:58.372853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 74/146] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:58.373739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.448273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.530306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9a34-9ad6-42f2-b355-89183f83c651 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:58.550677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c1ce7-7e0c-4d56-b0a4-6118bf6ce4cc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:58.603111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.603940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.604224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.604499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.620619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.624309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.624603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.624881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.629119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:58.645160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:58.646039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:58.646797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 73/147] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:58.647444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.697527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.778911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2076-ae65-40ef-8880-26d75f3857b1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:58.792853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:58.794300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:58.814042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.814850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.815134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.815409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.816658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:58.818199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:58.819906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 74/148] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:58.820847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.849477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.924272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:58.934983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17545f75-8629-4d73-9180-179e71e4395f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:58.952764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 75/149] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:42:58.953912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:58.976274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:58.977007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.977288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.977569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.978448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:58.978764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:58.980191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8a47e-b159-4136-af61-79634aae7d81 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:58.989532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 75/150] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:58.990602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e863-142c-475b-92c2-3aea3c0e5af2 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:58.990893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e863-142c-475b-92c2-3aea3c0e5af2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:58.991175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e863-142c-475b-92c2-3aea3c0e5af2 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:58.992902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 76/151] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:58.993620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e863-142c-475b-92c2-3aea3c0e5af2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.063567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c1d87-b6df-4c2b-8170-51213f651803 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.093987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.099721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.100739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.101188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.101731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.102783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.104146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e863-142c-475b-92c2-3aea3c0e5af2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.104840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1f54b-d2d4-481c-876c-7b46d6948268 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.135637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 76/152] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:42:58 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:42:59.137895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.138184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.138461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.167602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.168544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.168820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.169092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.170653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.172730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.176755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 77/153] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:58 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:59.179003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:59.180608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:59.181793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.224833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.333381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 77/154] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:59.334262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.366314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a96d1f-bd83-4446-badc-7bb4007cd67d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.408715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.409610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.409884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.410156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] PATH_INFO: `/v3/users/dc78cacc045349fabeddccb7d8c6557f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.411345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] RBAC: Authorizing `identity:get_user(user_id=dc78cacc045349fabeddccb7d8c6557f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.412950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.415921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 78/155] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/users/dc78cacc045349fabeddccb7d8c6557f => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:59.416825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.472240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881bd4f-14f7-4e63-88e0-1619b0776d85 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.491757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.513807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.519040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.519827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.520157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.520385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.521533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:59.522315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.524199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.532159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 78/156] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=8cde109646d74ea789591d2724109ef7 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:59.533574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.536432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148ced6-da5a-481d-9d91-9df6bcefc33e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.595030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.596293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.596774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.597256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.611449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7281eeee-f648-4972-8018-e83fc944b967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:42:59.620310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:42:59.621418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:42:59.622301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:42:59.624998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 79/157] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:42:59.626607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c9dc-4472-4e11-ad0e-9269c1d82b20 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.627089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c9dc-4472-4e11-ad0e-9269c1d82b20 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.627554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c9dc-4472-4e11-ad0e-9269c1d82b20 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.629730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 80/158] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:42:59.636230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e470-5347-4790-88a7-58717955625d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.636854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e470-5347-4790-88a7-58717955625d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.637380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e470-5347-4790-88a7-58717955625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.657761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2519e470-5347-4790-88a7-58717955625d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:42:59.659386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2519e470-5347-4790-88a7-58717955625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:42:59.676028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b07fa-bfcd-4b1e-adce-b7dd11481ecc None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.723498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.724430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.724712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.724986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.727718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.728597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.730735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 79/159] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:59.731980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.747685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 81/160] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:42:59.748914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519e470-5347-4790-88a7-58717955625d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.804288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.874097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:42:59.878014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57333e-630c-4195-8d8f-ef9cbd2b59aa None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.900800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519e470-5347-4790-88a7-58717955625d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:42:59.924620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.925309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.925592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.925865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.926637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:42:59.927172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.928446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.938896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 80/161] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:42:59.944265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:42:59.953269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:42:59.954565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:42:59.955044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:42:59.955563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/dc78cacc045349fabeddccb7d8c6557f/roles/a5aac4a009874b078e7dffc9863a98f0` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:42:59.958039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, user_id=dc78cacc045349fabeddccb7d8c6557f, role_id=a5aac4a009874b078e7dffc9863a98f0)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:42:59.959754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:42:59.979180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 82/162] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:42:59 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/users/dc78cacc045349fabeddccb7d8c6557f/roles/a5aac4a009874b078e7dffc9863a98f0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:42:59.980543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.122951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489a4d9-120d-41bf-aad8-54cf8ce49023 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.125360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e55a6-0c8b-448a-867c-5b80f137ab07 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.144185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.174254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.175085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.175359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.175630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.178463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.180193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.180731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.181210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] PATH_INFO: `/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.183001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] RBAC: Authorizing `identity:get_user(user_id=e56c84bcaa9a49c8bd5f12e10aeab889)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:00.184733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb61fe-91a8-4344-b7de-92cbfca8097b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:00.187691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 83/163] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/users/e56c84bcaa9a49c8bd5f12e10aeab889 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:00.189278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fabb9-86da-41f5-8c51-95ebc7da2a26 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.189905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fabb9-86da-41f5-8c51-95ebc7da2a26 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.190389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fabb9-86da-41f5-8c51-95ebc7da2a26 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.191994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 84/164] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:00.192917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:00.199382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.199672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.199982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:00.202269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:00.203113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:00.203933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 81/165] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:42:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:00.205664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.226489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.238201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:00.239438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:00.323587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 85/166] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:00.324692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.339333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f36fb-026b-45df-b3a2-137e89350acd None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.383534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.384830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.385346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.385800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.448271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.455418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2b99e-9219-4f19-8539-217ca0e83bd5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.464322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:00.465899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:00.519055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.519734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.520042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.520310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.521161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:00.521463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:00.522858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:00.532215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.544680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 86/167] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:00.546995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.654619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 82/168] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:00.655419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.675020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911492-3710-43b6-bd75-5bf508f6c30e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.710586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.712102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.712590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.713062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.737875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:00.745813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:00.746824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:00.747633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 87/169] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:00.748325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.820230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.865244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c83540-1b84-4b00-954e-28fb1614b711 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:00.876268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:00.932719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:00.933514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:00.933788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:00.934056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:00.935437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:00.936948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:00.938616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 83/170] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:00.944503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:00.953282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4c03f-616b-4bf9-854b-08e7a9eceeec None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.016629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.017502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.017778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.018049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.020742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.022207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.023954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 88/171] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:01.026059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:01.140230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f155-48b2-4f3b-a837-cc207e0a6fe0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.209275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.209892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.210171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.210444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.211219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:01.211586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.214033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.236651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb6f-f481-4ba0-8129-7df0ca1217ac None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.245795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 84/172] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:01.249399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:01.276225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:01.282067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:01.297274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.297925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.298204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.298478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.421388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.422874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.426200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b2e19-fdfa-4499-b1d3-1c2dc90de8a2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.489534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.490130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.490412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.490687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.491536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=e56c84bcaa9a49c8bd5f12e10aeab889) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:01.491850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.494026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.512384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 85/173] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:01 2022] GET /identity/v3/role_assignments?user.id=e56c84bcaa9a49c8bd5f12e10aeab889&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:01.513065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:01.651575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:01.706965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 89/174] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:01.713837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:01.723937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce618e1-542f-4e5c-aae5-21205de37560 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.788393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:01.797003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.797926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.798208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.798486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] PATH_INFO: `/v3/users/36056ea955f24614b41b93e8556945e6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.800584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] RBAC: Authorizing `identity:get_user(user_id=36056ea955f24614b41b93e8556945e6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.802941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.805291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 86/175] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:01 2022] GET /identity/v3/users/36056ea955f24614b41b93e8556945e6 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:01.811599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:01.943384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30a29-a482-4b8b-a7a0-24faf03a30f5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:01.989052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:01.990307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:01.990806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:01.991275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:01.992856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:01.994591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:01.996584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 90/176] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:01 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:02.004870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:02.077965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089bd41-c952-4e47-ae0c-855f72ae17f6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:02.156250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:02.167641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367633-1f92-467e-9063-377d5bb5bea4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:02.183902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e381a-382f-4209-994b-40d9256f173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:02.183902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e381a-382f-4209-994b-40d9256f173e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:02.183902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e381a-382f-4209-994b-40d9256f173e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:02.183902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e381a-382f-4209-994b-40d9256f173e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:02.185136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:02.228614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:02.229301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:02.229585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:02.229861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:02.230724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:02.231045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:02.232569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:02.239176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 91/177] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:02 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:02.319553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e381a-382f-4209-994b-40d9256f173e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:02.321198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e381a-382f-4209-994b-40d9256f173e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:02.461403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 87/178] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:43:02.468998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:02.635597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd67067-33c1-402c-a7ac-3c92df4e432b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:02.652356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caca3-407c-4650-af8a-256a899b1437 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:02.652993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caca3-407c-4650-af8a-256a899b1437 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:02.653462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caca3-407c-4650-af8a-256a899b1437 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:02.656557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 88/179] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:02.661218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:02.661752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:02.662215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:02.681193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:02.682748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:02.695915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:02.696964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:02.697249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:02.697531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] PATH_INFO: `/v3/projects/030c170c7f324776ab88181f1fc096d9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:02.698786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] RBAC: Authorizing `identity:get_project(project_id=030c170c7f324776ab88181f1fc096d9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:02.700749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:02.702497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 92/180] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:02 2022] GET /identity/v3/projects/030c170c7f324776ab88181f1fc096d9 => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:02.710576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:02.752344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:02.767729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:02.817630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 89/181] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:02.842938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:02.945989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb60ad4-3eb0-4a26-bd43-47c0044a3db6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:02.985875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d580170-26b9-4c98-b5c9-e3572bf24718 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.000842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.001502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.001786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.002064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.002962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=030c170c7f324776ab88181f1fc096d9, user.id=36056ea955f24614b41b93e8556945e6) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:03.003218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.004729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98952-c7ae-42ff-adf4-ae84621b9d0c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.011343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 93/182] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:02 2022] GET /identity/v3/role_assignments?user.id=36056ea955f24614b41b93e8556945e6&scope.project.id=030c170c7f324776ab88181f1fc096d9&role.id=8cde109646d74ea789591d2724109ef7 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:03.012661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840562-9cb9-48d8-a154-eba729b912bb None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.012935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840562-9cb9-48d8-a154-eba729b912bb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.013212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5840562-9cb9-48d8-a154-eba729b912bb None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.014721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 94/183] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:03.021479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94e3f7-f882-4a08-b067-29555fa5a211 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.021836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94e3f7-f882-4a08-b067-29555fa5a211 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.022059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94e3f7-f882-4a08-b067-29555fa5a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.039031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.040488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.042428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.042972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.067943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fc94e3f7-f882-4a08-b067-29555fa5a211 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:03.069916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fc94e3f7-f882-4a08-b067-29555fa5a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:03.070826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:03.088635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:03.089618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:03.095649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 90/184] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:03.101234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.163631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.197127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 95/185] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:03.208782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef6316-55c3-471f-b076-027764754b23 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.209317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef6316-55c3-471f-b076-027764754b23 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.209767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef6316-55c3-471f-b076-027764754b23 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.211374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 96/186] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:03.216909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.217392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.218175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.238177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:03.239764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:03.256927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.274411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6ba2b-7907-4e43-bea7-17b7a9ef8960 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.314926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.315600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.315907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.316180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.316931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:03.317236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.317823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 97/187] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:03.318639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f617-6dc0-4893-9b06-a73a37f758c2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.318733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.332226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 91/188] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:03.332226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c228ac-f56b-4e5c-93ef-85a913f1562a None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.332226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c228ac-f56b-4e5c-93ef-85a913f1562a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.332226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c228ac-f56b-4e5c-93ef-85a913f1562a None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.332226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 92/189] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:03.333297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c228ac-f56b-4e5c-93ef-85a913f1562a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.421500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.423154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504ce3e-d42a-4c83-8966-cab577edc581 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.439240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c228ac-f56b-4e5c-93ef-85a913f1562a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.457197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.458082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.458432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.458773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.470672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:03.474576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.475450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.475751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.476044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-5dbf2e10-89a5-4028-aee2-883f9abfdc9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:03.478431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:03.479328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:03.480275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 98/190] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:03.481024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.481308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.481607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.494477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.496009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.497772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 93/191] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:03.498447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:03.499620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:03.501772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.503307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.569281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 99/192] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:03.569838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.603165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe9af-257a-4340-983f-42c6c761ff5e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.639604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.640465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.640743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.641013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.642182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.643663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.645345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 94/193] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:03.646102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.660131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.667413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af687e51-4937-448f-aa3b-78e33090a00b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.701385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.702048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.702329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.702607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.703346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:03.703653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.705108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.713342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 100/194] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:03.714130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.717480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.746431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcd48e-4b91-4a87-8e45-144c1cf504ae None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.780717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.781367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.781642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.781910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.813520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ce16-195b-4e0c-b685-dc03769f1723 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:03.846282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:03.847018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:03.847299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:03.847577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:03.848219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:03.849689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:03.857663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:03.863091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:03.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:03.866637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:03.867411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 101/195] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:03.872651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.934839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:03.950002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 95/196] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:03.950855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:03.977219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a58c51-05ea-4a86-9971-de2c1f2f54af None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.009956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.010681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.010967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.011243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.012000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.013475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.034142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 102/197] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:03 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:04.037102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.052739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16e664-0750-4b21-b955-60444b551937 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.068821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.094273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.095577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.096128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.096567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] PATH_INFO: `/v3/users/dc78cacc045349fabeddccb7d8c6557f` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.097878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=dc78cacc045349fabeddccb7d8c6557f)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.099452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.106529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 96/198] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:03 2022] GET /identity/v3/users/dc78cacc045349fabeddccb7d8c6557f => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.107251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.154465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99591e5-8804-4163-9f28-1a5afd2b41ec None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.173087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.193900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.194574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.194852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.195124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.195998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:04.196464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.198066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.206335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 103/199] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.208528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.228882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d98ac7-002f-4d35-8bb5-fefe64675bf1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.268784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.269333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.269616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.269893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.271107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.272844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.274687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 97/200] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.275490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.335183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.344855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e8cd2-ba06-4473-9023-1eaaf8d4c1ad None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.386350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.387021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.387302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.387577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.401409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412be9-7570-4d24-84ac-d6da28a75e86 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.443039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.444454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.445047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.445508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.447361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.447900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.449940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.451893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 98/201] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.452918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.472732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.474695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.579647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0f602-ddc7-41b4-8a66-6ce45421bf0d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.592196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 104/202] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:04.593349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.605713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.630197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.631631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.631940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.632219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.633444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=21347b46a1674f9d803a1bfaecc573eb) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:04.633779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.635943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.643242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 99/203] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/role_assignments?user.id=21347b46a1674f9d803a1bfaecc573eb&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.643749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.713001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:04.737569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc36be5-33ca-4757-8e96-d5ad59480968 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.778481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.779594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.780142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.780650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.811952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97478a-4d66-4f23-8002-5d2b8bc326fa None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:04.893783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:04.894601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:04.894887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:04.895161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:04.899956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.900817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:04.902086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.902302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:04.904435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 100/204] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:04.910129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:04.968413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:05.007645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:05.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88115de3-9877-4547-a96e-3ead7cc16410 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:05.031371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 105/205] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:05.037929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:05.059747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:05.060443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:05.060739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:05.061002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:05.062308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e, user.id=dc78cacc045349fabeddccb7d8c6557f) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:05.062631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:05.064263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:05.070730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 101/206] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:04 2022] GET /identity/v3/role_assignments?user.id=dc78cacc045349fabeddccb7d8c6557f&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:05.228338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1124da2-2989-4543-8879-ce76e968a024 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:05.278502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:05.279376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:05.279651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:05.279961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:05.281159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:05.282640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28dfb32-7187-4e33-9e73-05412bb5e390 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:05.284528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 106/207] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:05 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:05.291631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:05.400464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f26fec-63be-4aa8-8c74-06d0c3ddf4b6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:05.416299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:05.441531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:05.442263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:05.442596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:05.442877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:05.443726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=21347b46a1674f9d803a1bfaecc573eb) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:05.444070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:05.445611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27968982-a3bb-4d69-808d-1a47b86daf69 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:05.452190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 102/208] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:05 2022] GET /identity/v3/role_assignments?user.id=21347b46a1674f9d803a1bfaecc573eb&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:06.345425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fed6-e347-4f9b-a6a2-f7a6ef84d236 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.345923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fed6-e347-4f9b-a6a2-f7a6ef84d236 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.345923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82fed6-e347-4f9b-a6a2-f7a6ef84d236 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.347457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 107/209] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:06.353020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.353822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.354289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.372134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:06.373629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:06.378609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9581df-243f-46b4-85af-2212f094ed79 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.379113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9581df-243f-46b4-85af-2212f094ed79 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.379575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9581df-243f-46b4-85af-2212f094ed79 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.381199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 108/210] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:06.387609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.387913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.388330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.405937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:06.407104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:06.454898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 103/211] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:06.469453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:06.486533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:06.496144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 109/212] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:06.520077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:06.611747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7ff54-a6a9-4df2-9f44-1a6c4dcede4f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:06.620663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:06.636598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c90bf6-b3a6-452c-87ee-8bc86e2be9fe None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:06.654195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:06.655408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.655944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.656417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.658073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:06.659809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:06.662113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 104/213] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:06.667594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:06.678463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:06.679541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.680034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.680617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.693078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:06.701554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:06.702443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:06.703237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:06.705999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 110/214] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:06.712240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:06.757382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:06.792841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca63e7d-dbe2-4bd3-ad38-698b4068ff6d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:06.828644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:06.829664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.830152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.830613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.831549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:06.833307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bdd9b-4f0a-448d-8aa2-4eb53c3d28fa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:06.842212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b48253-8740-40e7-844e-922f22fcc30b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:06.861134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 105/215] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:06 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:06.892472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:06.893454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.893877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.894293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.895280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:06.895747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:06.897374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:06.908007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 111/216] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:06.911978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090ffd-b253-46bc-b81e-42cdc5e5759e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.911978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090ffd-b253-46bc-b81e-42cdc5e5759e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.911978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090ffd-b253-46bc-b81e-42cdc5e5759e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.915500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 106/217] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:06.918190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:06.920709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:06.921000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:06.921276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:06.943549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:06.944800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:06.972685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.022018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 107/218] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:07.040712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.043593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e000e-e270-4637-a767-3e38b80c21ca None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.047932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.082535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.083579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.083866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.084168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] PATH_INFO: `/v3/users/a35eee1930274cbe8db32ee32c0bdb4c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.100647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] RBAC: Authorizing `identity:get_user(user_id=a35eee1930274cbe8db32ee32c0bdb4c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.102421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.104398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 112/219] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:06 2022] GET /identity/v3/users/a35eee1930274cbe8db32ee32c0bdb4c => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:07.110197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.176367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7ab98-c1dd-45ff-8820-c000e347a646 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.222677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.223487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.223765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.224065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.236786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:07.245822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:07.246758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:07.247562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:07.248715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c50151-69f3-4e50-adcc-40c37f7e385d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.252234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 108/220] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:07.253552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.279865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.291101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.292032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.292446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.292895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.325744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.380279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.382554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.419721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfab4-6ae2-4c55-8ffd-6bade9ff1330 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.489610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.490209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.490491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.490767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.490985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 113/221] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:07.491531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:07.491828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.494009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.496595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.512371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 109/222] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:07.517770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.600478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4833b2-bb44-4e5d-bf3a-0a21f1912c65 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.623798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.645547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.646947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.647426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.647951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.649976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.651744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a612e0-ede1-4bd5-a1aa-f4e2593a6c80 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.654222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 114/223] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:07.655825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dd25-9c53-41e4-88e8-8332294d835e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.656349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dd25-9c53-41e4-88e8-8332294d835e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.656811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028dd25-9c53-41e4-88e8-8332294d835e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.658460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 115/224] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:07.659755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028dd25-9c53-41e4-88e8-8332294d835e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.663238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.682280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c82d0-a27f-4a85-88c7-c7edb5f93fa5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.716155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.717039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.717365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.717712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.718980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.720529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3614c4-3479-4416-aec6-8aab02e76855 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.726770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 110/225] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:07.728181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.728467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.728742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.745505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:07.747586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:07.777157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028dd25-9c53-41e4-88e8-8332294d835e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.805651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.816910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.817252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 111/226] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:07.818179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.818706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.819187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.820247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=a35eee1930274cbe8db32ee32c0bdb4c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:07.820792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.822535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:07.824777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.829627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 116/227] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/role_assignments?user.id=a35eee1930274cbe8db32ee32c0bdb4c&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=8cde109646d74ea789591d2724109ef7 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:07.861163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:07.931117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:07.955332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a78ccb-fdf9-4706-bd33-d290b8e33dd3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:07.994227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:07.995216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:07.996055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:07.996353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:07.997723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:07.999267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.001484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 112/228] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:08.008397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:08.096847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037faefa-953a-4e58-852f-8f7a388ecf6e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.122743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2465f0b-a57d-47dc-bc6e-99e0b017c92a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.140477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.141846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.141846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.141846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.158328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:08.166755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.167713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.168022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.168296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/users/21347b46a1674f9d803a1bfaecc573eb/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:08.170153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:08.171024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:08.171825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:08.172465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, user_id=21347b46a1674f9d803a1bfaecc573eb, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:08.172465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.174618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 117/229] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:08.181258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:08.217340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 113/230] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:08 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/users/21347b46a1674f9d803a1bfaecc573eb/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:08.236850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:08.297299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241da9da-fef8-4801-abf4-8d348cb74b94 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.334699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.335750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.336239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.336738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.337784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:08.338278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:08.339892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.349345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 118/231] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:08.355401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:08.367059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:08.500344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b8076-25fd-4b90-bc6b-41d00754f17a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.542485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.543438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.543714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.544013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.547104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:08.548671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.550631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 114/232] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:08 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:08.556142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:08.690142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6332d-840f-41ae-98ea-192c8e294269 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.703739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:08.725925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.727208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.727679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.728163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.729575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:08.731224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd275093-0c1f-4e99-ba59-d5e1fe77b216 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.733173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 119/233] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:08 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:08.739177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:08.819454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:08.844002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ec35-7c7c-451a-bf9d-3deb030f39d6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:08.880035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:08.881305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:08.882101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:08.882591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/users/21347b46a1674f9d803a1bfaecc573eb/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:08.885089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, user_id=21347b46a1674f9d803a1bfaecc573eb, role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:08.886765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61e8f-2685-408e-9c67-ac3e6748c052 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:08.903408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 115/234] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:08 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/users/21347b46a1674f9d803a1bfaecc573eb/roles/a0d79cab834140ccb511ceb372f735a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:09.588415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d153e3-b7ca-4a0d-90a5-9e9860753c26 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:09.588964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d153e3-b7ca-4a0d-90a5-9e9860753c26 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:09.588964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d153e3-b7ca-4a0d-90a5-9e9860753c26 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:09.590633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 120/235] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:09.596555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fbe2f-07bf-4c56-b21e-27a73489f107 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:09.597081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fbe2f-07bf-4c56-b21e-27a73489f107 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:09.597542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fbe2f-07bf-4c56-b21e-27a73489f107 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:09.599262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 121/236] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:09.600074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:09.600861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:09.601322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:09.603770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:09.604307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:09.604769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:09.632691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:09.636308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:09.639414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:09.642875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:09.766184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 116/237] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:09.768939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 122/238] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:09.788446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:09.803655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:09.915290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:09.966818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afab60-8ac2-437a-b0f3-157a0b54c8b4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:09.979347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.007851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88462-4581-434c-995e-f65ef3849e2d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.013998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.015112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.015594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.016091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.038007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:10.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:10.049130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:10.049917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:10.052602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 123/239] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:10.063374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.090646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.091459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.091739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.092032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.093310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:10.094796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:10.096479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 117/240] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:10.101946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.235279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322740c-8ac6-4ac5-9a7f-41cf93934eb9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.270492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.271171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.271451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.271723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.272599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:10.274227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:10.284194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dae6e-6bee-4666-bc36-0372ea7a42ba None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.284349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.301209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 118/241] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:10 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:43:10.342658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.343418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.343747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.344065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.344848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:10.345208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:10.346855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74598177-f531-435b-b0c4-51f32187a227 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:10.356745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 124/242] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:10.362736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.491442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69c22-ff24-47c8-939e-999347c50f0a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.505166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a5641-1438-4466-8f70-d55a4b755472 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.505461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a5641-1438-4466-8f70-d55a4b755472 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.505747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a5641-1438-4466-8f70-d55a4b755472 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.507766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 125/243] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:10.513666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.513959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.514239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.530501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.531731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.531918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:10.532335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.532748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] PATH_INFO: `/v3/users/5c971e027e5b4a48b5e84140161d2c6d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.533109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:10.548902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] RBAC: Authorizing `identity:get_user(user_id=5c971e027e5b4a48b5e84140161d2c6d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:10.550761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:10.552792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 119/244] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:10 2022] GET /identity/v3/users/5c971e027e5b4a48b5e84140161d2c6d => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:10.558499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.565041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.621268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 126/245] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:10.646532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.672244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.693927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08797612-92a3-4fba-a1d4-caef65eb1f70 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.734041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.734708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.734990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.735265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.763240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad98b2b-6eba-419e-ae75-5d7ec64e4cbd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.799303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:10.800162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.800612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.800891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.812108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:10.813605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6f1a5-c948-454d-840a-3cd13b96c8a4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:10.814592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-cb8a945e-0d44-4bb4-b5da-517f27048081 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:10.822890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:10.823798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:10.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 127/246] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:10.825862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49df5d-dfb1-4ab5-bd2b-4a8ee409490b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.825862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49df5d-dfb1-4ab5-bd2b-4a8ee409490b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.826129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49df5d-dfb1-4ab5-bd2b-4a8ee409490b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.832555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 128/247] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:10.833479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49df5d-dfb1-4ab5-bd2b-4a8ee409490b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:10.862001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.868577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:10.929306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 120/248] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:43:10.931243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:10.931778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:10.933035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:10.954852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49df5d-dfb1-4ab5-bd2b-4a8ee409490b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:10.957041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:10.959305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:11.005068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.005930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.006217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.006491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.008201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:11.012293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.014339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.026493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 129/249] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.027559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.047203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 121/250] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:11.050035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.142769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:11.148320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:11.164492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00416a4c-96a3-4df1-a0b4-ad03a678e2fe None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.175664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81220be0-d92d-4874-85fb-b4b53c757904 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.213032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.214313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.214781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.215238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] PATH_INFO: `/v3/users/a35eee1930274cbe8db32ee32c0bdb4c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.216706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] RBAC: Authorizing `identity:get_user(user_id=a35eee1930274cbe8db32ee32c0bdb4c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.218441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.220688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 122/251] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/users/a35eee1930274cbe8db32ee32c0bdb4c => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.221884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.249415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.250085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.250368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.250639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.256941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.258490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.260468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 130/252] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.261064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.331173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdf52d-5175-46fe-9028-eb736ac2a834 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.367119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.368629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.369107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.369642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.385359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:11.399940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:11.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:11.401636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:11.406045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 123/253] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:11.407749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.428258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:11.488721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170284b3-2442-400a-9584-78ec92be7419 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.557717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.558599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.558879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.559155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.560783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.562270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.564205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 131/254] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.564824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.588464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b277e-a378-4fae-9b35-eec7709d3a39 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.591596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:11.626760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.627891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.628213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.628587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.629573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=5c971e027e5b4a48b5e84140161d2c6d) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:11.630143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.632718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb652e-d44d-4a2f-8835-957cc7776207 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.643964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 124/255] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/role_assignments?user.id=5c971e027e5b4a48b5e84140161d2c6d&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.645393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193af9-6514-4f9b-9f56-58029b8e435b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.645684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193af9-6514-4f9b-9f56-58029b8e435b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.645963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9193af9-6514-4f9b-9f56-58029b8e435b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.647368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 125/256] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:11.649195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9193af9-6514-4f9b-9f56-58029b8e435b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.707824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421c8f-beb3-404f-b5bc-0d678e9218e4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.775756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.776483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.776765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.777040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.777820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:11.778129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.779559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708deb5-bce9-49bb-811f-741f3fb1782a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.797075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 132/257] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:11.798622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.798864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.799142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.799319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:11.836721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:11.838022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:11.851952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9193af9-6514-4f9b-9f56-58029b8e435b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:11.923966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:11.925742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:11.926025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:11.926302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/users/a35eee1930274cbe8db32ee32c0bdb4c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:11.930701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, user_id=a35eee1930274cbe8db32ee32c0bdb4c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:11.934867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:11.954390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 126/258] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:11 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/users/a35eee1930274cbe8db32ee32c0bdb4c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:11.955649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:11.980592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 133/259] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:12.090652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.101354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a09794-376b-431b-aa88-d36a3c46c652 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:12.125832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:12.134795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:12.141749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:12.142659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:12.142939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:12.143215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:12.145000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:12.147064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:12.149303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 127/260] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:11 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:12.157413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.342565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf3c4e-8b16-4908-adbb-8a7bfff105f5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:12.395091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df96c-fe88-4d24-8546-c293bbd8f72f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:12.397857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:12.398693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:12.398977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:12.399409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:12.422125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:12.431403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:12.432410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:12.433200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:12.441509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 134/261] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:12.448045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.493282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:12.494096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:12.494256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:12.494446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:12.580262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:12.583475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:12.591537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:12.593080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:12.625693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8269-4634-4002-b211-1453255b790a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:12.711335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:12.712604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:12.712604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:12.712604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:12.713362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:12.713675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:12.715096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:12.732742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 135/262] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:12.738050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.743011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 128/263] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:12.748412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.847701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b592b-810a-4316-8ea6-12d0341bc768 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:12.860275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:12.884658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:12.885539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:12.885820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:12.886095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:12.890741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:12.890741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:12.890741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 129/264] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:12.896325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:12.922250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11671e3-eb54-4afb-9fbe-97c8b4ffd30e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.000619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.001946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.002420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.002880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] PATH_INFO: `/v3/users/21347b46a1674f9d803a1bfaecc573eb` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.003649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354f3c-b8c9-4e0b-a4bb-8ab954699ecb None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.004512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] RBAC: Authorizing `identity:get_user(user_id=21347b46a1674f9d803a1bfaecc573eb)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.006417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:13.008736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 136/265] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/users/21347b46a1674f9d803a1bfaecc573eb => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:13.017545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:13.027107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.043810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.044515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.044796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.045071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.045917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=21347b46a1674f9d803a1bfaecc573eb) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:13.046230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.047698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d4ab2-5c84-4556-a232-9dde3143dd56 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:13.054205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 130/266] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:12 2022] GET /identity/v3/role_assignments?user.id=21347b46a1674f9d803a1bfaecc573eb&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:13.202662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339ea8f-a915-468c-9751-e744569e955f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.249777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.250436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.250769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.250991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.336520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.338109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:13.340181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.340881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3242-0313-4d07-9915-9322f16bb87d None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.341173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3242-0313-4d07-9915-9322f16bb87d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.341450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d3242-0313-4d07-9915-9322f16bb87d None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.343044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 131/267] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:13.348582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.348867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.349150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.379504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:13.381198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:13.436233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.460804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 137/268] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:13.465889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:13.537363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 132/269] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:13.581693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:13.587924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264afb3e-9d2f-4315-a4fd-9a9a5b34e8cc None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.588153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.634756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.635629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.635931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.636208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.637401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.638879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380745e8-51f5-4960-9164-e81fd5724884 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:13.640643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 138/270] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:13 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:13.641823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751630f-e37e-469d-a153-beb1767e5252 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.642109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751630f-e37e-469d-a153-beb1767e5252 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.642386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751630f-e37e-469d-a153-beb1767e5252 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.643620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 139/271] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:13.647088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751630f-e37e-469d-a153-beb1767e5252 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:13.736510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878011e-9f5e-4de3-9645-57c5aee605ef None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.773773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751630f-e37e-469d-a153-beb1767e5252 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.783593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.785057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.785564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.785959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.800540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-fcc55f23-1fd4-4b55-922d-5e506ec53acd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:13.812106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:13.813217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:13.814198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:13.816020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 133/272] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:13.817720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.818005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.818847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.821094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.822117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.822683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.823160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.824216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=21347b46a1674f9d803a1bfaecc573eb) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:13.824740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.826397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:13.833428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 140/273] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:13 2022] GET /identity/v3/role_assignments?user.id=21347b46a1674f9d803a1bfaecc573eb&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:13.834438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:13.845056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:13.846273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:13.868495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.873111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:13.947396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26e274-8795-418a-a2e7-97f72fc596ac None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:13.977903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 134/274] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:13.986783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:13.987849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:13.988343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:13.988803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:13.989746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:13.990256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:13.991946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.002412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 141/275] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:14.008150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.020261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.121326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:14.131086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3113-c89f-4cea-9dc8-939262843b34 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.166342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.167504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.168002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.168484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.169920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:14.171567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.173768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 142/276] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:14.178610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c77f-7bd2-4ad5-bece-946d9abb6687 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.179038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.217428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.218369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.218660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.218959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] PATH_INFO: `/v3/users/e4c07cfbaee24209a434629747d913b2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.234150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=e4c07cfbaee24209a434629747d913b2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:14.235837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.238116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 135/277] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/users/e4c07cfbaee24209a434629747d913b2 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:14.243586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.258862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:14.291589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbb1d2-3e6a-4974-886e-c7a0c603bc85 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.334131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.335048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.335554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.335987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.336998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:14.338689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8390e12-3b89-4740-8907-4a3afe8d76b2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.368685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 143/278] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:14 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:43:14.377681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ff70-214f-4ef6-bfcb-43b9cac6d7f6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.423309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.424023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.424309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.424586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.425517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, user.id=e4c07cfbaee24209a434629747d913b2, scope.system=all) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:14.425833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:14.427425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee4cd-291a-484f-8314-31f0538a9a7f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.434696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 136/279] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/role_assignments?user.id=e4c07cfbaee24209a434629747d913b2&scope.system=all&role.id=8cde109646d74ea789591d2724109ef7 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:14.447045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd96c60-7140-4718-81db-90e360d085d7 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.447352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd96c60-7140-4718-81db-90e360d085d7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.447780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd96c60-7140-4718-81db-90e360d085d7 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.449581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 144/280] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:14.455706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10c05-8564-4431-b82e-ff3dca227f19 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.456310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10c05-8564-4431-b82e-ff3dca227f19 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.456726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10c05-8564-4431-b82e-ff3dca227f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.476020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cac10c05-8564-4431-b82e-ff3dca227f19 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:14.477599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cac10c05-8564-4431-b82e-ff3dca227f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:14.561728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 137/281] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:14.584917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd96c60-7140-4718-81db-90e360d085d7 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.613070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:14.702136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd96c60-7140-4718-81db-90e360d085d7 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.738312 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.739121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.739401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.739672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.747714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba58e-9bf0-48f2-a919-d902d3cb0261 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.748176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba58e-9bf0-48f2-a919-d902d3cb0261 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.748610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba58e-9bf0-48f2-a919-d902d3cb0261 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.750701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 138/282] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:14.756197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.757098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.757634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.764866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:14.769577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:14.773528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:14.776540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:14.777364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 145/283] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:14.783154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.792326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:14.793639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:14.871215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:14.887026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 139/284] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:14.893164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e41b-52a6-462a-b320-6b755114e37e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:14.911073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:14.951815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:14.952635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:14.952852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:14.953121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:14.953892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:14.954193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:14.955613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:14.975476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 146/285] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:14.982516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.073981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332a0ee-2aec-46bb-b6c1-c1f84bb66fa7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.110529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9a3c9-f437-4b26-83ab-2a3b0b2daa34 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.140241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:15.147018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.147926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.148200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.148468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] PATH_INFO: `/v3/users/5c971e027e5b4a48b5e84140161d2c6d` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.149682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] RBAC: Authorizing `identity:get_user(user_id=5c971e027e5b4a48b5e84140161d2c6d)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.151140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:15.153160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 147/286] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/users/5c971e027e5b4a48b5e84140161d2c6d => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:15.156540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.157356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.157635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.157904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.158603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.171980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:15.192844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:15.208879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:15.209774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:15.215448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 140/287] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:15.224911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.273054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efa48d-159c-4437-86b7-90970b6165bd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.307589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.308783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.309230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.309666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.311770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.313437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:15.319669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 148/288] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:15 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:15.326587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.444182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a265ff-751a-4ace-b722-7c9a9b0e5b9b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.472066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:15.509392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2c71-e5ca-4234-b694-84b308d362ac None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.515342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.516246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.516694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.517209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.518133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:15.518633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.520224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:15.534464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 141/289] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:15.540582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.601244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:15.640681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.641516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.641861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.642067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/users/5c971e027e5b4a48b5e84140161d2c6d/roles/a5aac4a009874b078e7dffc9863a98f0` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.647322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b72b6d-06a0-4aa7-948b-db85e10c4250 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.652698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, user_id=5c971e027e5b4a48b5e84140161d2c6d, role_id=a5aac4a009874b078e7dffc9863a98f0)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.654182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:15.686486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.687912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.688405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.688843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] PATH_INFO: `/v3/users/a35eee1930274cbe8db32ee32c0bdb4c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.690425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] RBAC: Authorizing `identity:get_user(user_id=a35eee1930274cbe8db32ee32c0bdb4c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.692126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:15.693215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 149/290] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:15 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/users/5c971e027e5b4a48b5e84140161d2c6d/roles/a5aac4a009874b078e7dffc9863a98f0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:15.694597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 142/291] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:15 2022] GET /identity/v3/users/a35eee1930274cbe8db32ee32c0bdb4c => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:15.709074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:15.863133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8f149-9872-4225-ae11-61b65fa8e9cf None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:15.898124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:15.898792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:15.899072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:15.899392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:15.909121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:15.981651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:15.983149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:16.090381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 150/292] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:16.095752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:16.206193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be04d-b378-487f-9e8a-b06f757be532 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:16.242747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:16.257820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:16.258782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.259063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.259338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.260593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:16.262085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5d726-5b56-49df-8f68-10546a3f7125 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:16.264162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 143/293] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:16 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:16.272590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:16.324248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:16.398896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7526-5dda-4b62-b484-3be890535b68 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:16.433242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:16.433900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.434179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.434452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.435284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=a35eee1930274cbe8db32ee32c0bdb4c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:16.435597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:16.437099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1ecfb-4f67-4125-b5cb-e1e4881cbc16 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:16.443465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 151/294] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:16 2022] GET /identity/v3/role_assignments?user.id=a35eee1930274cbe8db32ee32c0bdb4c&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=8cde109646d74ea789591d2724109ef7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:16.862522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb574-67a6-493f-9c7f-c01d431ac6e1 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.863028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb574-67a6-493f-9c7f-c01d431ac6e1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.863316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abb574-67a6-493f-9c7f-c01d431ac6e1 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.865021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 144/295] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:16.868347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c029a-4759-4b58-9ace-9f351ce3fe69 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.868807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c029a-4759-4b58-9ace-9f351ce3fe69 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.868986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c029a-4759-4b58-9ace-9f351ce3fe69 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.870303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 152/296] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:16.874938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.875230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.875514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.882218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:16.882503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:16.882781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:16.898559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:16.899867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:16.906492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:16.907648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:16.987961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 145/297] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:16.995001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 153/298] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:17.005270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.023001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.056542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.113868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978ccc-a385-4016-b9e6-72a55bdd9a71 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.128932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50eb58-6cfc-47e9-9e3e-4f1432f6fd99 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.136117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.152688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.154271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.154963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.155605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.164705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.165852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.166135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.166412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.170000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:17.177667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:17.178241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:17.179086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:17.179862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:17.182511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 146/299] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-46902740-b290-421b-b53b-97e51b128c37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:17.187917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:17.188828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:17.189629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 154/300] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:17.192012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c15f3-4607-41bd-abaa-df9404394b9d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.192939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c15f3-4607-41bd-abaa-df9404394b9d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.193225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c15f3-4607-41bd-abaa-df9404394b9d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.194694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 155/301] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:17.199239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c15f3-4607-41bd-abaa-df9404394b9d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.296222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.304834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bde82e-f7b6-4bae-a18d-7cbaee5bcb43 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.306718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c15f3-4607-41bd-abaa-df9404394b9d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.340843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.341518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.341802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.342079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.342830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:17.342920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.343141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.343517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.343802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.344105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.344575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4fe78-6542-421d-84ac-39e744fd1b06 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.344855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:17.345159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.346866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.358119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 156/302] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.358119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.358119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.358119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.361108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 147/303] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.362039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.375759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:17.376915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:17.410688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.443664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 157/304] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:17.444437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.478070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d5f7d-f3be-409a-bb6e-41e796012de9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.522709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.525244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.525690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.526112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] PATH_INFO: `/v3/users/640f1a97358f45419d86f8058f5ef0d1` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.539494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.548090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=640f1a97358f45419d86f8058f5ef0d1)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.549579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.551309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 148/305] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/users/640f1a97358f45419d86f8058f5ef0d1 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.552110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.564251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3f213-ec7b-44ac-85a8-23c3391e8f58 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.605000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.606067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.606347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.606664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] PATH_INFO: `/v3/users/e4c07cfbaee24209a434629747d913b2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.607909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] RBAC: Authorizing `identity:get_user(user_id=e4c07cfbaee24209a434629747d913b2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.609378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.615717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 158/306] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/users/e4c07cfbaee24209a434629747d913b2 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.615717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.665019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.668618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9c371-bae1-4040-a56f-0fe02bb31bd0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.705647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.706577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.706874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.707151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.708809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.710283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.712183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 149/307] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.717470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.720353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3b1b2-48c0-4418-bad4-4a6248f4407f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.756826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.757952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.758118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.758575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.759913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, user.id=640f1a97358f45419d86f8058f5ef0d1, scope.system=all) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:17.759913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.762160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.768212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 159/308] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Jul 14 10:43:17 2022] GET /identity/v3/role_assignments?user.id=640f1a97358f45419d86f8058f5ef0d1&scope.system=all&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:17.768846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:17.788513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:17.855317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123bf14-5cfe-4e6b-9c13-216c7c16be9a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.884633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba32f-d427-40bf-ac77-638e98594314 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:17.895238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.896521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.897046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.897509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] PATH_INFO: `/v3/system/users/e4c07cfbaee24209a434629747d913b2/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.899403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e4c07cfbaee24209a434629747d913b2, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.901146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee7542-84b4-47ec-84f8-a7a572b72370 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.914900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 150/309] 10.222.0.27 () {58 vars in 1334 bytes} [Thu Jul 14 10:43:17 2022] PUT /identity/v3/system/users/e4c07cfbaee24209a434629747d913b2/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:17.930482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:17.931174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:17.931459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:17.931733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] PATH_INFO: `/v3/groups` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:17.932544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:17.934137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33629c-8a6b-498d-959f-89b53900b979 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:17.958761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 160/310] 10.222.0.27 () {60 vars in 1134 bytes} [Thu Jul 14 10:43:17 2022] POST /identity/v3/groups => generated 229 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:18.218135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754bb49-fbb9-44a5-ba33-7259551603f4 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.218631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754bb49-fbb9-44a5-ba33-7259551603f4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.218631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754bb49-fbb9-44a5-ba33-7259551603f4 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:18.220285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 151/311] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:18.225676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.225977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.226261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:18.243670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:18.244901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:18.314964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 161/312] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:18.338910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754bb49-fbb9-44a5-ba33-7259551603f4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:18.373828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:18.449540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754bb49-fbb9-44a5-ba33-7259551603f4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:18.485241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:18.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.486240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.486509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:18.497433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:18.505177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:18.506232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:18.507421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 152/313] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:18.513062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:18.521670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:18.618100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7280e4a-ccdf-4ace-924f-c0d0a68fa7b5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:18.653674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:18.654605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.654886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.655151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:18.655915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:18.656223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:18.657731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:18.666631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 162/314] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:18.672478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:18.771096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:18.780294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233ac67-36d5-482e-822d-f6afc5b09731 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:18.814191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:18.815405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.815911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.816406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] PATH_INFO: `/v3/users/5c971e027e5b4a48b5e84140161d2c6d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:18.817800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] RBAC: Authorizing `identity:get_user(user_id=5c971e027e5b4a48b5e84140161d2c6d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:18.819457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:18.821761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 153/315] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:18 2022] GET /identity/v3/users/5c971e027e5b4a48b5e84140161d2c6d => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:18.831897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:18.900986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:18.957407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac09f31-b538-4f7b-b974-72d7a5f58385 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:18.997026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:18.998095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:18.998572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:18.999074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:19.072680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:19.074497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:19.150400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:19.191618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 163/316] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:19.197333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:19.305277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b04ed-996a-45a1-a677-ec9d072cf5ad None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:19.338650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:19.339446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:19.339724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:19.340044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:19.341226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:19.342684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc42a4-f013-4c28-93fd-8047e84e957f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:19.344642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 154/317] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:19 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:19.350406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:19.448398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8d3ac-aa57-4d6b-b2f5-6abf3ad8d44e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:19.487944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:19.488958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:19.489433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:19.489921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:19.490957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, scope.project.id=b4b181899df5484c91638c9809cb7222, user.id=5c971e027e5b4a48b5e84140161d2c6d) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:19.491480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:19.493175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc13e2-30d0-4d19-a247-98cf28f41e40 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:19.500445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 164/318] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:19 2022] GET /identity/v3/role_assignments?user.id=5c971e027e5b4a48b5e84140161d2c6d&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:19.903786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343326d8-66d9-4a9b-9864-43fa4289c659 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:19.904057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343326d8-66d9-4a9b-9864-43fa4289c659 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:19.904353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343326d8-66d9-4a9b-9864-43fa4289c659 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:19.905999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 155/319] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:19.911249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afdd9-ace7-43cf-b956-62990ee8c7bd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:19.911702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afdd9-ace7-43cf-b956-62990ee8c7bd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:19.911796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afdd9-ace7-43cf-b956-62990ee8c7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:19.929218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-777afdd9-ace7-43cf-b956-62990ee8c7bd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:19.930421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-777afdd9-ace7-43cf-b956-62990ee8c7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:19.965328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.010048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 165/320] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:20.032548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343326d8-66d9-4a9b-9864-43fa4289c659 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.062568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.082554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59122-3ae0-422c-83e2-f238db76a77c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.082903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59122-3ae0-422c-83e2-f238db76a77c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.083085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59122-3ae0-422c-83e2-f238db76a77c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.084501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 166/321] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:20.089334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba80ca-8c01-4e9a-a04e-12407d6387da None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.089615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba80ca-8c01-4e9a-a04e-12407d6387da None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.089890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba80ca-8c01-4e9a-a04e-12407d6387da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.107446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ddba80ca-8c01-4e9a-a04e-12407d6387da None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:20.108687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ddba80ca-8c01-4e9a-a04e-12407d6387da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:20.140486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343326d8-66d9-4a9b-9864-43fa4289c659 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.174791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 167/322] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:20.175420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.176207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30345a-989d-4539-bd3c-8a316eaed15b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.176346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.176519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30345a-989d-4539-bd3c-8a316eaed15b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.176609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.176767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30345a-989d-4539-bd3c-8a316eaed15b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.176867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.178171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 168/323] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:20.183375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.183658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.184000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.189606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:20.197393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:20.198296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:20.199094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 156/324] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:20.199751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.201348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:20.202539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:20.266575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 169/325] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:20.267511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.286527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.295084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.299614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2ab9-6503-4175-a14e-7998efadc968 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.332256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.333066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.333348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.333623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.334804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.336314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.337997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 157/326] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:20.340720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.371171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f2e32-7ec8-413d-8563-2282e4da7cfc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.404975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.405728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.406010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.406279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.407018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:20.407331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.408733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.416928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 170/327] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:20.417685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.438283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dcece-ca5b-4954-9f25-1d9877c33a1b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.471457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.472285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.472567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.472882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.483293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:20.490848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:20.491782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 158/328] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:20.497769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.502607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.512626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.520029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d43bca-60f0-43c3-8e71-3d7ea868c7b9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.552616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.553274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.553554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.553837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] PATH_INFO: `/v3/groups` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.554507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.556056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.572090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 171/329] 10.222.0.27 () {60 vars in 1134 bytes} [Thu Jul 14 10:43:20 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:20.572904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.600705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07664bb2-0ae9-4485-8ff5-47a41727d341 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.642199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.643101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.643376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.643643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] PATH_INFO: `/v3/users/640f1a97358f45419d86f8058f5ef0d1` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.644979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] RBAC: Authorizing `identity:get_user(user_id=640f1a97358f45419d86f8058f5ef0d1)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.646489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.648427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 159/330] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/users/640f1a97358f45419d86f8058f5ef0d1 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:20.654343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.703768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9248d-d768-49f0-b8da-9429adb53e66 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.743793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.744544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.744834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.745104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.745852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:20.746160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.747632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.750468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.756607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 172/331] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:20.765066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:20.771763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8135e-9438-480c-bfca-3b71442c35e6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.774225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:20.812589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.814601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.814601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.814857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] PATH_INFO: `/v3/system/users/640f1a97358f45419d86f8058f5ef0d1/roles/a5aac4a009874b078e7dffc9863a98f0` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.817331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=640f1a97358f45419d86f8058f5ef0d1, role_id=a5aac4a009874b078e7dffc9863a98f0)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.819191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.835490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 160/332] 10.222.0.27 () {58 vars in 1334 bytes} [Thu Jul 14 10:43:20 2022] PUT /identity/v3/system/users/640f1a97358f45419d86f8058f5ef0d1/roles/a5aac4a009874b078e7dffc9863a98f0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:43:20.894433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981e30b-5821-4663-92ac-a1786ff04972 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:20.942965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:20.943955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:20.944196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:20.944470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] PATH_INFO: `/v3/users/e4c07cfbaee24209a434629747d913b2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:20.945793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] RBAC: Authorizing `identity:get_user(user_id=e4c07cfbaee24209a434629747d913b2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:20.947464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4244-7cbf-43cc-86b5-ea292349a2ce None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:20.949392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 173/333] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/users/e4c07cfbaee24209a434629747d913b2 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:20.955199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:21.071241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc014d5-ceac-4985-9605-5e05bdc1e042 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:21.108988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:21.108988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:21.109439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:21.109555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:21.110417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] RBAC: Adding query filter params (role.id=8cde109646d74ea789591d2724109ef7, user.id=e4c07cfbaee24209a434629747d913b2, scope.system=all) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:21.110727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:21.112182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d469d-a765-4d5d-a68e-bc21f2f01937 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:21.124858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 161/334] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Jul 14 10:43:20 2022] GET /identity/v3/role_assignments?user.id=e4c07cfbaee24209a434629747d913b2&scope.system=all&role.id=8cde109646d74ea789591d2724109ef7 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:23.114304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98be2d-ea14-462c-9d58-f07f2c47899a None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.114805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98be2d-ea14-462c-9d58-f07f2c47899a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.114878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98be2d-ea14-462c-9d58-f07f2c47899a None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.116481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 174/335] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:23.122347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f5b4e-ef2b-40cd-9a93-1f0308d94cf7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.122802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f5b4e-ef2b-40cd-9a93-1f0308d94cf7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.122876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f5b4e-ef2b-40cd-9a93-1f0308d94cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.136118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:23.144859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-da3f5b4e-ef2b-40cd-9a93-1f0308d94cf7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:23.146052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-da3f5b4e-ef2b-40cd-9a93-1f0308d94cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:23.233530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 162/336] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:23.259779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98be2d-ea14-462c-9d58-f07f2c47899a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:23.346021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1558d9-4412-4960-8904-4d1b92002307 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.346346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1558d9-4412-4960-8904-4d1b92002307 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.346568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1558d9-4412-4960-8904-4d1b92002307 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.347986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 163/337] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:23.354976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef830-5e5e-441e-944b-d35ec081d1ba None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.355251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef830-5e5e-441e-944b-d35ec081d1ba None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.355530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef830-5e5e-441e-944b-d35ec081d1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.356676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:23.383425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-229ef830-5e5e-441e-944b-d35ec081d1ba None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:23.384740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-229ef830-5e5e-441e-944b-d35ec081d1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:23.393068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98be2d-ea14-462c-9d58-f07f2c47899a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:23.457511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:23.458330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.458614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.458895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.488371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:23.500421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:23.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:23.502507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 175/338] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:23.507903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:23.514512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 164/339] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:23.536880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e210-6890-49cf-b8f1-b91cf83eda21 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.537531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e210-6890-49cf-b8f1-b91cf83eda21 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.537531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e210-6890-49cf-b8f1-b91cf83eda21 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.539293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 165/340] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:23.545930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.546762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.547234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.576895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:23.578106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:23.706107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:23.721492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 166/341] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:23.722205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:23.722537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92c68d-d64f-4a5b-aa6b-44e684f72fa1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:23.770440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:23.770440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.770440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.770440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.770985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:23.771551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:23.772717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f7124-6684-42b9-9d34-8748185175cf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:23.774150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:23.783560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 176/342] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:23.787272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a246cf-01ae-4781-94bb-784360b9ca5d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.787848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a246cf-01ae-4781-94bb-784360b9ca5d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.788378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a246cf-01ae-4781-94bb-784360b9ca5d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.790126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 177/343] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:23.791136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a246cf-01ae-4781-94bb-784360b9ca5d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:23.854912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a33e9c-7034-4ed3-b912-9166197d510d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:23.892077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:23.892939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.893282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.893475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.898186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a246cf-01ae-4781-94bb-784360b9ca5d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:23.905047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:23.914830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:23.916268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 167/344] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:23.917112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:23.964252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:23.965070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:23.965351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:23.965617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:23.980917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2781ec8e-8779-457f-bb7a-61da6f587ba4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:23.997249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:23.998197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:23.999014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 178/345] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:24.000575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.003944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.004278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.011040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.033107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:24.035150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:24.069148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1373044-a7d4-4d02-905a-00068a5db30e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.101668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.121899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.122917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.123459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.123459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.125613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 179/346] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:24.126560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.201238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.202715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.236270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491564a-9383-43f1-b719-1f0b2939e328 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.258178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.275109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.275784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.276090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.276366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.277115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] RBAC: Adding query filter params (name=reader) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:24.277420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.278883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.288303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 180/347] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:24.289039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.304461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 168/348] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:24.305330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.340204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.395736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262549-d758-44ff-82a4-8b0ca821a311 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.413491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc1d38-755c-4f0c-a9dd-d92774431533 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.436323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.437006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.437289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.437563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.438885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:24.439200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.441051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.457027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 181/349] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:24.458413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.471104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.471945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.472228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.472499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.489334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:24.498441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:24.499538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:24.500525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 169/350] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:24.501319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.528935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.597108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82aebee-d85d-4e89-bd4b-d0cd48d4f30d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.610237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.640451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927458-2a87-4548-b225-bcdccd5744a7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.642497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.644476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.644973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.645473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] PATH_INFO: `/v3/users/640f1a97358f45419d86f8058f5ef0d1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.646838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] RBAC: Authorizing `identity:get_user(user_id=640f1a97358f45419d86f8058f5ef0d1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.649253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.652002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 182/351] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/users/640f1a97358f45419d86f8058f5ef0d1 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:24.653339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.684029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.685191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.685867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.686276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.688305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.690294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.693405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 170/352] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:24.694619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.771346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393634d-e8fe-481d-b0a4-3c53573653a2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.800029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.809716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.810744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.811354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.811646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.820881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a6bf8-98a3-4d72-8430-fe046d1373e9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:24.857662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:24.858342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:24.858622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:24.858893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:24.859641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:24.859968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.860494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:24.861358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.872640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 171/353] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:24.873351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:24.895099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:24.896743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:24.998754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c651-e2ec-43aa-871f-b5e0476be68b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.032446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 183/354] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:25.033124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.043344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.044225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.044330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.045760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.045760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] RBAC: Adding query filter params (role.id=a5aac4a009874b078e7dffc9863a98f0, user.id=640f1a97358f45419d86f8058f5ef0d1, scope.system=all) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:25.045760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.047590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61df25e-0c8a-478d-ac7e-fd59e129eb81 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.061394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 172/355] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Jul 14 10:43:24 2022] GET /identity/v3/role_assignments?user.id=640f1a97358f45419d86f8058f5ef0d1&scope.system=all&role.id=a5aac4a009874b078e7dffc9863a98f0 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.062511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1bf5-d803-48a6-b808-3f4a280a296b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.062793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1bf5-d803-48a6-b808-3f4a280a296b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.063067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1bf5-d803-48a6-b808-3f4a280a296b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.064669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 173/356] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:25.065609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1bf5-d803-48a6-b808-3f4a280a296b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.078812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.126930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.165314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8490e3-11c5-41fd-8918-37ae092cf2ba None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.205030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1bf5-d803-48a6-b808-3f4a280a296b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.211272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.213169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.213615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.214032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.221045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.222559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.224580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 184/357] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.226178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.254429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.255102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.255414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.256015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.340249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.341715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8835d50-e7be-48a9-8fd1-ccc3398c7bab None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.344433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.356906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596c6c-3f8a-4704-8e42-ee674945d118 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.400310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.400751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.401033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.401371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.403536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.405053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf261-96f3-4f01-b0a0-42d5e074469e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.411253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 185/358] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.411253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.411253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.411253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.427984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:25.429234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:25.433066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.472979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 174/359] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:25.475059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01411-effd-4e8b-ae4d-d6c464d48e46 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.475059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01411-effd-4e8b-ae4d-d6c464d48e46 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.475059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01411-effd-4e8b-ae4d-d6c464d48e46 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.477844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 175/360] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:25.477844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01411-effd-4e8b-ae4d-d6c464d48e46 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.515350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 186/361] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:25.516642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.596558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01411-effd-4e8b-ae4d-d6c464d48e46 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.599176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.638491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.639561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.639839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.640173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.640978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7053f-8752-4bb0-9fc2-69fe83a65e57 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.641095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:25.641318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.642779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.650097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 176/362] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=8cde109646d74ea789591d2724109ef7 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.651103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.687034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.687952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.688230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.688500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.689760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.691322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dcad7-258d-4d64-964f-19b49fed030c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.696654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 187/363] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.696654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.696654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.696654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.714644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:25.715901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:25.718174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.791639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652106ba-f94c-4649-b1e5-a4120d333e37 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.815469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 188/364] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:25.817250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.833810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.834831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.835207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.835576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:25.836970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:25.838837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:25.840713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 177/365] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:25.841577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:25.878901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:25.946086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76558b1a-a75a-449c-b96e-1431f5da7b65 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.959526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cda2-b26d-4001-9836-d406d9059c6e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:25.997952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:25.998782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:25.999060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:25.999329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.007988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.008659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.009515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.010026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.010916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.011781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:26.012182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.013274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:26.013770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:26.022420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:26.023398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:26.024348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 189/367] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:26.025153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 178/367] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:25 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:26.025832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.026999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.157510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a92bf-b373-43da-ac90-76758c852592 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.167706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.174430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b427-d037-4c7d-931b-beccd2112d66 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.205249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.206583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.207061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.207550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.220801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:26.222896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.223780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.224094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.224368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.225565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.227025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.229023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 179/368] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:26.229564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:26.229668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:26.230491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:26.232046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 190/369] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:26.237048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.323863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.360608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe15b-65fe-42dd-b128-66b806950461 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.376527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c287e8e-eec5-4a1e-a1ce-6f281e37a2ca None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.407153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.408435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.409144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.409596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.411132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:26.411618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.414166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.418292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.419050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.419401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.419752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.420629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:26.421003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.422565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.426054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 180/370] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:26.427153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.436677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 191/371] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:26.437514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.467707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.547513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5696533-aae8-4ff9-a2e1-4221c563557c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.547706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43855043-b710-4bd7-8d1f-a54df470443e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.569185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.584652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.584869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.585508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.585791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.586066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.586497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.587406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.587902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.588961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=b4b181899df5484c91638c9809cb7222) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:26.589467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.591132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.597418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 181/372] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=8cde109646d74ea789591d2724109ef7 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:26.598609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.660700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.662247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.702392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.710156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb324f4-43da-44ed-a0c8-916aff34cc41 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.747862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:26.748894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:26.749398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:26.749894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:26.778201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 192/373] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:26.784144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:26.809171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.831065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:26.833149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:26.930133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0510a2d-f2da-4944-8f9f-6207b3562b65 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:26.946014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:26.950573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 182/374] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:26.956668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:27.012791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.013596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.013885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.014152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.020657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.022151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.024260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 193/375] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:27.031346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:27.124682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515aad6e-8ae1-435d-be03-63cf7dec3087 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:27.204235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:27.213167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.214365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.214846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.215295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] PATH_INFO: `/v3/projects/dd0314351a554271b0b9af50b68ac028` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.220111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b59e0-7335-4a47-9094-fa0f4a18adcf None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:27.220821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] RBAC: Authorizing `identity:get_project(project_id=dd0314351a554271b0b9af50b68ac028)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.222527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.224791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 183/376] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:26 2022] GET /identity/v3/projects/dd0314351a554271b0b9af50b68ac028 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:27.230020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:27.267319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.268164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.268448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.268731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.270589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.272123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.273788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 194/377] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:27 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:27.280335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:27.370299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:27.423504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e07ede-e0ad-462d-83fe-78061eab91d3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:27.463766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.464857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.465137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.465410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] PATH_INFO: `/v3/groups/50d8d6e3883f498db224681089adbd10` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.471916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] RBAC: Authorizing `identity:get_group(group_id=50d8d6e3883f498db224681089adbd10)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.473399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.477968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 184/378] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:27 2022] GET /identity/v3/groups/50d8d6e3883f498db224681089adbd10 => generated 229 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:27.482747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:27.500052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700fe3b-2345-4f54-ad8d-19a1968bb326 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:27.600684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.601282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.601567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.602731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=b4b181899df5484c91638c9809cb7222) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:27.603049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.604566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bffcd-3ff6-4e44-8031-08167ba507ea None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.610301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fca28-2f6f-4ef8-9ed9-b1bb64be4d8f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:27.622324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 195/379] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:27 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:27.652310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:27.660622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:27.661441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:27.661732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:27.662013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:27.662864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] RBAC: Adding query filter params (group.id=50d8d6e3883f498db224681089adbd10, role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=dd0314351a554271b0b9af50b68ac028) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:27.663177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:27.664771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5945ec-c93c-41ed-9d71-a3a1d8e3a079 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:27.676084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 185/380] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:27 2022] GET /identity/v3/role_assignments?group.id=50d8d6e3883f498db224681089adbd10&scope.project.id=dd0314351a554271b0b9af50b68ac028&role.id=bd93d91744f54f739650facbdec5aef5 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:28.074167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d45be-5ba6-4152-a607-4b6e252f6aff None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.074879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d45be-5ba6-4152-a607-4b6e252f6aff None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.075364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d45be-5ba6-4152-a607-4b6e252f6aff None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.077193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 196/381] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:28.082942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.083470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.083972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.108940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:28.110504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:28.198694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 186/382] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:28.233411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d45be-5ba6-4152-a607-4b6e252f6aff None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:28.259335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:28.366382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d45be-5ba6-4152-a607-4b6e252f6aff None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:28.405698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:28.406512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.406798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.407073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.424101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:28.431962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:28.433001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:28.434372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 197/383] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:28.439257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:28.588233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ef78-3ad6-4a07-9691-2482a2047906 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:28.611263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:28.625100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:28.625777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.626407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.626680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.627433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:28.627737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:28.629295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c83a-87ed-4e7f-a178-caf7d3c7d2b8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:28.638974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 187/384] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:28.644625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:28.734247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:28.755495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420ea4-e7e0-49ef-9357-2cb2574d87d3 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.756144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420ea4-e7e0-49ef-9357-2cb2574d87d3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.756621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27420ea4-e7e0-49ef-9357-2cb2574d87d3 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.758824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 188/385] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:28.762041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2ba52-7266-4b19-a38d-9e902abc7d39 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:28.765687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.767134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.767612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.794348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:28.795920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:28.826774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:28.827652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:28.827957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:28.828228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:28.829614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:28.831080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:28.841668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 198/386] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:28 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:28.843048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:28.935636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 189/387] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:28.986599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:28.997812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39c9a9-324c-4da5-927a-20e09502daad None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.036992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.038136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.038559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.038967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.040486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:29.042618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:29.045387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 199/388] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:28 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:29.051466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.092327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:29.166325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9c572-f54e-4d39-ba8c-1fff68d0e225 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.183394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536929b9-1fd4-41f8-9cdb-d801f0bc07b8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.185793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:29.209991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.211109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.211580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.212020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.219935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.220834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.221108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.221377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/groups/1101c91e5bcc439baa48b6669549e3ac/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.223466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, group_id=1101c91e5bcc439baa48b6669549e3ac, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:29.224409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:29.224954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d92890-c21a-4e19-8938-58298a425491 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-10a53584-0182-4f60-bb47-c03a5281cecb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:29.233310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:29.234456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:29.235511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:29.248008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 200/390] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Jul 14 10:43:29 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/groups/1101c91e5bcc439baa48b6669549e3ac/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:29.248008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678e91-ea17-42c0-a627-cd88db4fafc9 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.248008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678e91-ea17-42c0-a627-cd88db4fafc9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.248008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678e91-ea17-42c0-a627-cd88db4fafc9 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.248843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 190/391] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:29.250058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 201/391] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:29.252052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.252729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.253183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.255268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d678e91-ea17-42c0-a627-cd88db4fafc9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.271278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:29.272815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:29.388095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 191/392] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:29.404293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:29.424477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.476465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d678e91-ea17-42c0-a627-cd88db4fafc9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.528074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.528751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.529027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.529594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.530347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:29.530677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:29.532957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:29.549457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 202/393] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:29.551138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.573679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ba034-1293-473b-8cb3-05e6c7fc79bb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.574522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:29.608848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.609655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.609934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.610203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] PATH_INFO: `/v3/roles/member` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.620911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:29.628837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:29.629703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:29.630674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 192/394] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:29.638321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.745216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3324ae-9965-432e-aaa1-4deaa19eaffa None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.749647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5e705-61fc-4050-b000-573d99442a93 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:29.794828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.795505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.795783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.796678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.797434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:29.797741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:29.799145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:29.801208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:29.802088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:29.802417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:29.802683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:29.803856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:29.805307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:29.807361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 203/395] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:29.813791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.819999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 193/396] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:29.825612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:29.825839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:29.872247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.004019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd9759-f3dc-4edd-af15-f0a8f5fdfd10 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.018689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873b44c-a743-4694-ab31-4326433dfea8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.068885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.069764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.070048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.070322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.072168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:30.073640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:30.077847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 204/397] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:30.084471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.084751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.085342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.085615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.085882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.087041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:30.088618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:30.090523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 194/398] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:29 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:30.097847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.184298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.223655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7ca23-1305-4fd1-83e9-d62d6906f533 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.254323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.266671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7286-0257-4a69-bf32-a2645e5321d8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.289662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.291904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.292330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.292803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/groups/1101c91e5bcc439baa48b6669549e3ac/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.295222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] RBAC: Authorizing `identity:create_grant(project_id=97a48aba46424f5fad9bb4e7a7131c4e, group_id=1101c91e5bcc439baa48b6669549e3ac, role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:30.296891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:30.309528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.310903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.311259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.311609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.313214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:30.314756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20751245-1885-4eb7-85e2-ef422b442be7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:30.319927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 195/400] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:30 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:30.319927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dc8bf-16ce-46e0-bb48-9d04097c6be4 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.319927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dc8bf-16ce-46e0-bb48-9d04097c6be4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.319927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dc8bf-16ce-46e0-bb48-9d04097c6be4 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.320445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 196/401] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:30.320585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 205/401] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Jul 14 10:43:30 2022] PUT /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e/groups/1101c91e5bcc439baa48b6669549e3ac/roles/a0d79cab834140ccb511ceb372f735a2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:30.331644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22448c3-2007-40c5-a43c-1066663b8c05 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.331644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22448c3-2007-40c5-a43c-1066663b8c05 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.331644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22448c3-2007-40c5-a43c-1066663b8c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.332024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.357075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b22448c3-2007-40c5-a43c-1066663b8c05 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:30.358566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b22448c3-2007-40c5-a43c-1066663b8c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:30.469248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 197/402] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:30.470564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a0ba-92c1-49a0-8bb6-bda698e18398 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.470866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a0ba-92c1-49a0-8bb6-bda698e18398 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.471125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405a0ba-92c1-49a0-8bb6-bda698e18398 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.472592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 198/403] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:30.478874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.479226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.479515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.498406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:30.499235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576b71e-df89-4f04-802d-3f410e0e71cd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.499678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:30.542756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.543687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.544273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.544273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/groups/1101c91e5bcc439baa48b6669549e3ac/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.546676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, group_id=1101c91e5bcc439baa48b6669549e3ac, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:30.548177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:30.551254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.568847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 206/404] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Jul 14 10:43:30 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/groups/1101c91e5bcc439baa48b6669549e3ac/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:43:30.572816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.583953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 199/405] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:30.625329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.674777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.761404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e00c2-14df-440a-98b8-3c7f45c0a25c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.806111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de957c6-505f-432b-a1a3-f9862c96b2d0 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:30.811234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.812149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.812433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.812706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.824786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:30.833051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:30.834063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:30.834861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 207/406] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:30.842423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.857660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:30.858620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:30.859005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:30.859376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:30.873872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:30.881265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:30.882318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:30.883722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:30.890510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 200/407] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:30.892230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:30.921923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:30.997555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25383323-8629-476b-bcb7-dc59f476852c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.048056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:31.068399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.069417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.069863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.070295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.071217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:31.072940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.074543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.086106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839cf0-8052-41d5-bbfb-d52f0731dd05 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.100173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 208/408] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:31.111244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.136276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.137281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.137741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.138177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.139256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:31.141151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.142737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.159668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 201/409] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:31.168861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.280318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29412189-9782-4b38-828e-2f7310a707d5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.311036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:31.341068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.342269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.342717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.343153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.344637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.346612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dda51-c964-4e1b-b23a-e2d573eff442 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.348289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.350442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 209/410] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:31 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:31.356560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.395668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.396575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.397137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.397417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] PATH_INFO: `/v3/groups/50d8d6e3883f498db224681089adbd10` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.398630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] RBAC: Authorizing `identity:get_group(group_id=50d8d6e3883f498db224681089adbd10)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.400239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.406853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 202/411] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:31 2022] GET /identity/v3/groups/50d8d6e3883f498db224681089adbd10 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:31.421245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.440942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:31.565164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d8a23-8fec-4b5f-8cae-b1c8a6395de7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.565954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d884dd5-1e96-4cc7-a4dd-028a92fc801f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.604902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.605871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.606156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.606478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] PATH_INFO: `/v3/projects/dd0314351a554271b0b9af50b68ac028` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.607852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] RBAC: Authorizing `identity:get_project(project_id=dd0314351a554271b0b9af50b68ac028)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.609647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.612420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 203/412] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:31 2022] GET /identity/v3/projects/dd0314351a554271b0b9af50b68ac028 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:31.618152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.643644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.644629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.644908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.645181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.652874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.654469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.656496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 210/413] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:31 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:31.662194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:31.709840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:31.733698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e531ad-2327-48e2-a6b4-d7600c220804 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.753359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:31.773081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.774458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.774971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.776176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] PATH_INFO: `/v3/projects/dd0314351a554271b0b9af50b68ac028/groups/50d8d6e3883f498db224681089adbd10/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.778333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] RBAC: Authorizing `identity:create_grant(project_id=dd0314351a554271b0b9af50b68ac028, group_id=50d8d6e3883f498db224681089adbd10, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.779977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762054d3-fc64-45d1-a2fc-51e13364458e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.799121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 204/414] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Jul 14 10:43:31 2022] PUT /identity/v3/projects/dd0314351a554271b0b9af50b68ac028/groups/50d8d6e3883f498db224681089adbd10/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:31.801239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775dd15-7f1e-4e7a-8e42-802d8678b2d7 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.802006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775dd15-7f1e-4e7a-8e42-802d8678b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.802502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775dd15-7f1e-4e7a-8e42-802d8678b2d7 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.809355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 205/415] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:31.811172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.811462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.813545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.820456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b3e-b28e-4f60-aa81-ba214dae4162 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:31.835536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:31.836795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:31.861723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:31.863658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:31.864333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:31.864808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222/groups/1101c91e5bcc439baa48b6669549e3ac/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:31.867523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4b181899df5484c91638c9809cb7222, group_id=1101c91e5bcc439baa48b6669549e3ac, role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:31.869891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:31.902803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 211/416] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Jul 14 10:43:31 2022] PUT /identity/v3/projects/b4b181899df5484c91638c9809cb7222/groups/1101c91e5bcc439baa48b6669549e3ac/roles/a0d79cab834140ccb511ceb372f735a2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:43:31.917146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 206/417] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:31.940441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:32.057482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9682b84-bad3-430e-aa07-696cefdcf118 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:32.072853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:32.095769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:32.096638 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.096921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.097192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.108869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:32.116809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:32.117769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:32.119188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 212/418] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:32.124616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:32.196294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:32.232041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899cdc0-b3b9-4628-9a28-87ff9d1b82c6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:32.271410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:32.272494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.273000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.273483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.274435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] RBAC: Adding query filter params (name=member) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:32.275277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:32.276903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:32.290927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 207/419] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:32.296466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:32.436574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ec1a9-3c54-4177-b1ab-c2f45b401c1b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:32.474508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:32.475167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.475449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.475723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.536191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:32.570048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:32.571551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097edb6-0a52-4771-8eef-2fb502fda2b3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:32.707107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 213/420] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:32.712489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:32.834316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623e79b-5c42-41be-955e-780a31b74edd None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.834639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623e79b-5c42-41be-955e-780a31b74edd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.834871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623e79b-5c42-41be-955e-780a31b74edd None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.836548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 214/421] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:32.841596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.841880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.842158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.873830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:32.875008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:32.886133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc63e-7409-4320-9199-5d4287a6ee06 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:32.945426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:32.957499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:32.958363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:32.958640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:32.960170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:32.962583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:32.965296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:32.971998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 208/422] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:32 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:32.973288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.015838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 215/423] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:33.051455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.056294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.122403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf8d4d-7dfa-4709-9917-d3cb388983c8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.160330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.161617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.162186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.162760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.164214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:33.166126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8879e-5118-4962-8bc9-a7f858e06f9d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:33.168498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 209/424] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:32 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:33.170062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14b6c-7aef-46ac-b1c2-b1bcbcde4075 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.170558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14b6c-7aef-46ac-b1c2-b1bcbcde4075 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.171030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14b6c-7aef-46ac-b1c2-b1bcbcde4075 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.172491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 210/425] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:33.178734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e14b6c-7aef-46ac-b1c2-b1bcbcde4075 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.184242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728c3c-9116-4018-9229-19a040930c1d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.238067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.239564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.239844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.240134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.254460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:33.256646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-f1d99a38-cb43-4540-a4a1-e54fc71fed77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:33.266709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:33.267661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:33.268959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 216/426] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:33.269900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.269900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.269900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.289578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:33.291352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:33.309102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e14b6c-7aef-46ac-b1c2-b1bcbcde4075 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.348909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.349573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.349851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.350122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.350966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:33.351274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:33.352770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:33.364106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.368195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 211/427] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=8cde109646d74ea789591d2724109ef7 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:33.369811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.379532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 217/428] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:33.407962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.527208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ab9c-8f6c-4331-a638-e9693530e193 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.569446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.571074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.571614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.572122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] PATH_INFO: `/v3/roles/member` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.597536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:33.607050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:33.608054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:43:33.608863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:33.616193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 218/429] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:33.621036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.635097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61721cb6-9c29-4ef3-a22d-8af193a98492 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.672255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.680218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.681435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.681720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.681999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.683141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:33.683452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:33.685523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:33.689127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.701987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 212/430] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:33.703296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.763198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407099b-a0aa-4a6a-bbd0-26e68262a526 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.802731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.803797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.804307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.804790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.805781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] RBAC: Adding query filter params (name=member) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:33.806495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:33.808145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:33.820827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 219/431] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:33.828077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:33.835959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb475-661f-4357-9780-8110dfae5b85 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.884021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:33.884590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:33.884874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:33.885196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:33.964868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.966537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a093-4122-40d0-93df-d6fbfd479929 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:33.983664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:33.986411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:33.988523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.011344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.012665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.013532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.014017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.093232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.094621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d6810-a81e-4ab5-ba52-5a8e16ff553b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.120795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 213/432] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:34.128150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.223542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 220/433] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:34.226199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1e34d-78f4-47fd-b16a-1341f35489de None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.226721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1e34d-78f4-47fd-b16a-1341f35489de None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.227500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1e34d-78f4-47fd-b16a-1341f35489de None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.229563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 221/434] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:34.230508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1e34d-78f4-47fd-b16a-1341f35489de None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.241417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.249681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6091ee5-05f8-458e-a0bd-f4acb738087d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.250584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.301709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.302945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.303221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.303497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] PATH_INFO: `/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.305574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] RBAC: Authorizing `identity:get_project(project_id=97a48aba46424f5fad9bb4e7a7131c4e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.307141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561b330-decc-47f9-88f8-ca44bf83a661 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.308900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 214/435] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/projects/97a48aba46424f5fad9bb4e7a7131c4e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.310083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.310331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.310602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.331176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:34.332393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:34.366461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1e34d-78f4-47fd-b16a-1341f35489de None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.402012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 215/436] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:34.402318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.402860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.403211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.403496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.403773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.404996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.406456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a18c0a-28d3-4024-867d-be78b994f14a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.408220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 222/437] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.409434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5405c-5e5b-4688-98fd-7913f593caf0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.409721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5405c-5e5b-4688-98fd-7913f593caf0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.409990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5405c-5e5b-4688-98fd-7913f593caf0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.411368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 223/438] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:34.416103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5405c-5e5b-4688-98fd-7913f593caf0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.495961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.497121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.507035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885a63f-8960-4fff-9db1-892e4ec496bb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.515359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5405c-5e5b-4688-98fd-7913f593caf0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.540708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.541550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.541823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.542092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.543250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.544776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73c196-f5c8-482d-a060-abc3982c79cf None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.546366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 216/439] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.547503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.547741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.548034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.548558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.549410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.549680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.549945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.551464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.552578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.554181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 224/440] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.554952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.564855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:34.566044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:34.631853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 217/441] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:34.632651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.664370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47834244-8d1f-4917-8a0e-373a7306a287 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.699296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.700140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.700408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.700672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.710750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:34.717718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:34.718726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:34.720137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 225/442] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:34.720926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.733613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:34.736620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84bf3a-5fc5-42b0-ba1f-dd6ab9014246 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.769774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.770422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.770695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.770961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.771792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:34.772123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.773562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.779487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 218/443] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=97a48aba46424f5fad9bb4e7a7131c4e&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.780492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.826390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be9df-3268-4d58-b116-3787692bd07c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.865499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.866471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.866894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.867308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.868505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=8cde109646d74ea789591d2724109ef7, scope.project.id=b4b181899df5484c91638c9809cb7222) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:34.869109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:34.870878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:34.878196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 226/444] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=8cde109646d74ea789591d2724109ef7 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:34.879412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:34.930774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732fe9e-073e-48bd-9662-83f5190988d8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:34.984859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:34.985688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:34.985969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:34.986240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:34.992524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.002540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:35.010833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:35.011802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:35.012651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:35.013558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 219/445] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:35.020167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.037084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe0f89-d610-4de6-8202-33d77017d027 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.042407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.078477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.079150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.079435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.079716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.081327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:35.081633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.083718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.104293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 227/446] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:35.107081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.154121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175313fd-1882-41cf-b8a0-aaa30c39c561 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.191689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.192614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.193103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.193554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.194327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:35.194637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.197559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.211040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 220/447] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:35.212403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.223383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da8732-534b-4d60-a1e0-115d6a7d3202 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.265723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.266704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.267148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.267396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.270977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.311499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.334775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ebf7-bc74-4e89-b642-8aa766c27472 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.354281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.355765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.371853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.372538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.372829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.373099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.443114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.444535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.456427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 228/448] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:43:35.462098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.496400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.524201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.551349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 221/449] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:43:35.557120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.563131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed5fe-3545-4e41-ba67-9c56e8433d42 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.596810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.597666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.597948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.598226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] PATH_INFO: `/v3/projects/b4b181899df5484c91638c9809cb7222` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.599400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b4b181899df5484c91638c9809cb7222)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.600903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.602590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 229/450] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/projects/b4b181899df5484c91638c9809cb7222 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:35.608666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.654654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ebbc-1949-4881-974e-7a612a3af8a2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.688860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.689721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.689999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.690269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] PATH_INFO: `/v3/projects/dd0314351a554271b0b9af50b68ac028` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.691570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] RBAC: Authorizing `identity:get_project(project_id=dd0314351a554271b0b9af50b68ac028)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.693079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.694864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 222/451] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/projects/dd0314351a554271b0b9af50b68ac028 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:35.700379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.708951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf604b5-d1dd-41ff-9536-4d88cd4189b5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.711236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.742279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.742679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.743668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.744524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.744843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] PATH_INFO: `/v3/groups/1101c91e5bcc439baa48b6669549e3ac` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.746062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] RBAC: Authorizing `identity:get_group(group_id=1101c91e5bcc439baa48b6669549e3ac)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.747618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.749287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 230/452] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/groups/1101c91e5bcc439baa48b6669549e3ac => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:35.755134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.800322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda17a4-cfef-4651-9d78-9de03c859343 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.835404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.836355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.836636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.836937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] PATH_INFO: `/v3/groups/50d8d6e3883f498db224681089adbd10` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.838409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] RBAC: Authorizing `identity:get_group(group_id=50d8d6e3883f498db224681089adbd10)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.839913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.841578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 223/453] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/groups/50d8d6e3883f498db224681089adbd10 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:35.847327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:35.855552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f72f38-e865-432e-bdb6-25d642ff5e79 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.889687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.890342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.890620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.890894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.891722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] RBAC: Adding query filter params (group.id=1101c91e5bcc439baa48b6669549e3ac, role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=b4b181899df5484c91638c9809cb7222) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:35.892067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.893501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2233-d417-4a35-87ba-e0f0cc070956 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:35.899557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 231/454] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/role_assignments?group.id=1101c91e5bcc439baa48b6669549e3ac&scope.project.id=b4b181899df5484c91638c9809cb7222&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:35.955967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f98688-7ccb-4d7f-912b-bf6f6203f173 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:35.968347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:35.995671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:35.996366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:35.996644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:35.996913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:35.997752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] RBAC: Adding query filter params (group.id=50d8d6e3883f498db224681089adbd10, role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=dd0314351a554271b0b9af50b68ac028) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:35.998093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:35.999544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1262d-fc5f-4e13-ae1f-c8609a5341a8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:36.006154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 224/455] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Jul 14 10:43:35 2022] GET /identity/v3/role_assignments?group.id=50d8d6e3883f498db224681089adbd10&scope.project.id=dd0314351a554271b0b9af50b68ac028&role.id=bd93d91744f54f739650facbdec5aef5 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:38.638232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c489a-6a6a-4911-a34c-fa18b7704fad None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.638770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c489a-6a6a-4911-a34c-fa18b7704fad None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.638770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c489a-6a6a-4911-a34c-fa18b7704fad None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.640217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 232/456] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:38.645552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db764a-ae1f-4e2e-9f13-533efaac9fb0 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.646379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db764a-ae1f-4e2e-9f13-533efaac9fb0 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.646850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db764a-ae1f-4e2e-9f13-533efaac9fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.668350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d0db764a-ae1f-4e2e-9f13-533efaac9fb0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:38.669873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d0db764a-ae1f-4e2e-9f13-533efaac9fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:38.761421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 225/457] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:38.764036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efa4e-6d0f-4c61-bd90-44b70789fb7c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.764568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efa4e-6d0f-4c61-bd90-44b70789fb7c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.765042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efa4e-6d0f-4c61-bd90-44b70789fb7c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.766831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 233/458] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:38.772030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144b4a-1872-41ef-a8fc-6d4a722fbe0e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.772322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144b4a-1872-41ef-a8fc-6d4a722fbe0e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.772602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe144b4a-1872-41ef-a8fc-6d4a722fbe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.785394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8efa4e-6d0f-4c61-bd90-44b70789fb7c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:38.801874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fe144b4a-1872-41ef-a8fc-6d4a722fbe0e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:38.803058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fe144b4a-1872-41ef-a8fc-6d4a722fbe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:38.824329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:38.878086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 226/459] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:38.879862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34f7f-089d-4c47-b183-2e297a284510 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.880300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34f7f-089d-4c47-b183-2e297a284510 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.880708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc34f7f-089d-4c47-b183-2e297a284510 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.882561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 227/460] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:38.893378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.893651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.893841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.904762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8efa4e-6d0f-4c61-bd90-44b70789fb7c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:38.916359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:38.918543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:38.954929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:38.956807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:38.957623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:38.957900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:38.958391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:38.971774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:38.973992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:39.001621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:39.002726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:39.003617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:39.003617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:39.003617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.003617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:39.003846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 228/461] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:39.005633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 234/462] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:39.008793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.014631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.112355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.124972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233edc5-8bcf-45a2-a0d9-4031a9bb7a1d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.131716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf7f74-5810-441f-a0ec-f1ea64118682 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.160563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.161438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.161919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.162365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.163254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:39.163727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.165419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.168779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.169587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.169862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.170128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.180778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.182542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.185382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 235/463] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:39.186414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:39.208212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:39.209110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:39.210025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:39.210025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:39.210025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.210025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:39.212723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 229/464] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:39.214285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.237291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.308563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f0425-a029-442f-89ef-1f1181a014c9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.333807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e20095-104f-46e7-a710-d60227c7a63d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.344480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.345598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.345894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.346173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.357048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.358832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.371164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.372495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.373307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.373592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.373866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.374578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.376071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f88c9-a992-4ab3-95a7-ea6afe5bdc1c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:39.381748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:39.382587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:39.383780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:39.383780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:39.383780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:39.383780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:39.388734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 236/465] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:39.389027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.401332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 230/466] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:39 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:39.403663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277060b-8b86-49da-8611-3ba5ecb5483f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.404236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277060b-8b86-49da-8611-3ba5ecb5483f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.404694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277060b-8b86-49da-8611-3ba5ecb5483f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.411539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 231/467] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:39.411539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277060b-8b86-49da-8611-3ba5ecb5483f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.505643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.524771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3c15d-8116-4cdb-99a5-1aa468149482 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.542776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277060b-8b86-49da-8611-3ba5ecb5483f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.566355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.567393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.567393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.567555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.568276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:39.568579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.569955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79beb7-692b-48cc-93ab-8cb81769bdce None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.585819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 237/468] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:39.588925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.589097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.589097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.589232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.589299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.589364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.589364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.589364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:39.589364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.590912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.605469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 232/469] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:39.606717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.609041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:39.610248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:39.686525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.700788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 238/470] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:39.702224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.750581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190970ad-d3c0-462f-9fd2-e16a9f20eb70 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.795085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:39.803096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.804057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.805024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.805514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.806764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.809141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.817143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a1d4f-08a4-4289-a90b-d7521611d8de None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:39.840610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 233/471] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:39 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:39.848883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:39.879262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:39.879991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.880273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.880544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.881257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:39.882734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778373a-5ee6-42cb-ab06-976c8c9125ad None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:39.917221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 239/472] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:39 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:39.918932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ad60-5757-49f0-b353-88ce58d18269 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.919220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ad60-5757-49f0-b353-88ce58d18269 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.919495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09ad60-5757-49f0-b353-88ce58d18269 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.921180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 240/473] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:39.937274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:39.937560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:39.937744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:39.971589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:39.973864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:40.013676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b583b07-1752-4f8b-82ec-1d21bb9206a7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.036345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:40.057740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.058787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.059137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.059906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.073251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:40.073561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 241/474] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:40.083651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:40.098092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:40.105739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:40.106646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:40.107473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:40.107473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:40.107473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:40.107473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:40.115794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 234/475] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:40.117643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:40.172449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:40.222428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d483f9c-b260-4096-aa43-4f1d86b99e2d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.248575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b89d6-9fc2-4300-a5f9-9e808755866f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.263168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.263850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.264567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.264867 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.265635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:40.267481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:40.275518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.sql.core [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:43:40.275518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:43:40.275518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [parameters: {'id': 'ed017419b93e490d87e5ab2ebbe5a451', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:43:40.275518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=95328) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result.read() Jul 14 10:43:40.298921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:43:40.299840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application yield resource Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise self.value Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application session.commit() Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:43:40.300884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:43:40.302114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:43:40.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result.read() Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:43:40.306517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [parameters: {'id': 'ed017419b93e490d87e5ab2ebbe5a451', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:40.308460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:43:40.317125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.318990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 242/476] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Jul 14 10:43:40 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:40.319109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.319252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:40.319384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:40.339044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 235/477] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:40.341151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:40.367701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:40.432156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d5f05-9224-4804-bb13-86e8633a5b1f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.468498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.469311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.469584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.469854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.483560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:40.484207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:40.492786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:40.493769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:43:40.494571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:40.495178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74dd21-5d79-47d7-a14e-fcca6595adce None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.499246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 243/478] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:43:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:40.501480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:40.564701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.565425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.565702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.566021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.566750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:40.568219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9b092-7406-42f0-aee9-610e712ee7da None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:40.610718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 236/479] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:40 2022] POST /identity/v3/users => generated 249 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:43:40.673925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:40.678993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c1bb3-73a1-431a-8f84-c84b2433f43d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:40.722682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:40.723676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:40.724121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:40.724590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:40.725789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:40.726263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:40.728714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fa686-38ff-46e7-b330-fe54a0c3f564 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:40.743641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 244/480] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:41.665860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359374-3a77-425f-bf26-5f063588a0d9 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:41.666927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359374-3a77-425f-bf26-5f063588a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:41.667412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359374-3a77-425f-bf26-5f063588a0d9 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:41.669297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 237/481] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:41.674937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:41.675473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:41.675993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:41.700900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:41.702421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:41.782215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 245/482] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:41.823750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359374-3a77-425f-bf26-5f063588a0d9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:41.975604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359374-3a77-425f-bf26-5f063588a0d9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.021993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.022807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.023089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.023360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.040824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:42.052661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:42.053608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:42.054406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 238/483] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:42.060145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.171843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:42.185102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f908f-3a83-4c3f-92c4-5ca1f2e8ae6e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.225545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.226155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.226439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.226714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.227494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:42.227802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:42.229409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:42.247503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 246/484] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:42.253026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.277757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:42.391484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3932e6-ee57-45ac-a195-a1a71c24b300 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.432619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.433703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.434139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.434553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.456190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:42.457938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-70b585b9-2604-4283-8db5-d540285d0d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:42.475764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:42.476811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:42.477855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:42.477855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:42.477855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:42.477855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:42.479748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 239/485] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:42.485570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.533421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb710e5-a609-43a1-b1c2-2de204f05ffd None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.533937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb710e5-a609-43a1-b1c2-2de204f05ffd None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.534356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb710e5-a609-43a1-b1c2-2de204f05ffd None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.535920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 240/486] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:42.540979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.541435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.541850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.564040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:42.566187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:42.641852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984a44f-7c8c-4e02-892f-bc1eea6ad2e2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.684322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.685245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.685668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.686114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.686978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:42.687425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:42.688998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:42.691008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:42.700280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 241/487] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:42.703071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 247/488] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:42.708383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.737729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.784249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:42.811301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40fd1e-4491-4650-9cc2-13aad2c16197 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.846250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.847047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.847481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.847905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.855839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:42.857517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-3fa5dc4c-fe6a-4531-b58f-0920fbe10414 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:42.873647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:42.874785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:42.875896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:42.878140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 242/489] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:42.879575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd5c8-7f91-4b88-963a-61e320415d26 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.880018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd5c8-7f91-4b88-963a-61e320415d26 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.880422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efd5c8-7f91-4b88-963a-61e320415d26 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.881837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 243/490] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:42.885439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efd5c8-7f91-4b88-963a-61e320415d26 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:42.899172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b22c7d-4d77-4ff1-9157-fdd567bf41e8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:42.946315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:42.947282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.948028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.948550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:42.960547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-79206412-28d2-4d6d-976e-0837c6f11288 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:42.969127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:42.970163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:42.970964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:42.972369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:42.973890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 248/491] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:42.974559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:42.975534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:42.975828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.002905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:43.005183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:43.026518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efd5c8-7f91-4b88-963a-61e320415d26 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.070945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.071984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.072398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.072764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.073912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:43.074211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.076090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.096533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 249/492] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:43.097851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.102748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.132034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 244/493] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:43:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:43.133858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.215993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b88a9-065d-4b9d-8a72-5c27a7b822a4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.241087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd05a6-7af9-41da-8cf5-38bb97f7c8f3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.251910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.253015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.253530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.254025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.254976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:43.255476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.257124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.260505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.267373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 250/494] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:43.268542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.280863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.281910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.282324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.282812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.293977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:43.302029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:43.302886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:43.303672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:43.306429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 245/495] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:43.307642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.338838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.381365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650f177-3ab8-4c37-9fbf-fd647a3e02fe None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.412730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d4982-3431-483f-b9ac-7e124b1cd1c1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.420878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.421544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.422125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.422125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.447759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.448661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.448938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.449208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.459367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.460863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:43.477231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:43.478122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:43.479036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:43.479036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:43.479036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:43.479036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:43.479036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 246/496] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:43.479648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.494193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.495922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6b106-6889-4556-bdb9-715e5348b510 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.506599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.561912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.580638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd8031-138e-422e-aa31-1e07d859c061 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.595840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 251/497] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:43.597226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a552a-38f8-4c07-88fb-b7fb4c9c2551 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.597508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a552a-38f8-4c07-88fb-b7fb4c9c2551 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.597781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a552a-38f8-4c07-88fb-b7fb4c9c2551 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.599050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 252/498] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:43.600109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691048b-dc1a-4f49-8532-07c0b13a22b2 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.600528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691048b-dc1a-4f49-8532-07c0b13a22b2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.600810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691048b-dc1a-4f49-8532-07c0b13a22b2 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.603914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 253/499] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:43.603914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691048b-dc1a-4f49-8532-07c0b13a22b2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.617918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.618590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.619240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.619578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.620358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:43.620661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.622028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.630793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 247/500] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:43.631552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.707546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691048b-dc1a-4f49-8532-07c0b13a22b2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.720900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.731335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d7355-71a8-4e81-9b70-81ea29c7d484 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.740252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.740909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.741194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.741462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.742178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:43.742481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.743931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a897-a6f2-4421-ac32-65ce093bf3a0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.754976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 254/501] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:43.756246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.756532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.756855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.768848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.769648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.769927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.770199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.772927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:43.774119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:43.781014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:43.782610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:43.794193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-96de198a-1b6f-4f53-ac52-48856d73a9bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:43.800166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:43.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:43.801898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:43.801898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:43.801898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:43.801898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:43.801898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 248/502] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:43.802557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.802819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.803093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.819509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:43.820714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:43.846127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 255/503] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:43.846953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.898064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 249/504] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:43.898905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:43.955005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac592f-098a-40be-bc67-0e388b96f729 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:43.975937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:43.988319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:43.989114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:43.989393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:43.989662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:43.999442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.000959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.005800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd2399-6cf8-4ee0-8800-ea74602f7cf6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:44.019273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.020300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 256/505] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.021361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.031908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.046457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.047262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.047543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.047817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.054938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.056549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:44.070839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.071799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 250/506] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.123188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96bca-2ad9-4fbc-8a31-5e824cf1593c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.158441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.159387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.159801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.160399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.161510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.161968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.164134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.170889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578e44-bfd2-4bd2-944a-e0cdd01176a2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.180097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 257/507] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:44.180882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.203916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.209441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.210239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.210516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.210784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.221306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.222765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:44.241194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.242231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 251/508] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.243313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.253445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.287318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103178bf-8946-4f2e-b874-d2270a6566a6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.321166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.322043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.322307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.322579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.332547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:44.340855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.341891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 258/509] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.342866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.347802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc292e2-e0f8-4428-a8d6-baca5ba3f2c3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.382887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.383898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.384182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.384558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.385598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.385899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.387280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.398507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 252/510] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:44.399452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.444867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.452832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcc944-d3b8-4a8e-846e-d7e5ad03e8d4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.477811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.485855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.486520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.486798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.487082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.487929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.488238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.489685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.499105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542a827-d3c7-4014-b74b-5c7bfda949a5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.526235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 259/511] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:44.528081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.536072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.536875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.537148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.537481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.544533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.546001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:44.564966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.565916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:44.566795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 253/512] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.567738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.629626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9b5d4-baa3-40bd-a565-d59a3d147744 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.665527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.666616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.666903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.667177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.667281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a8961-7904-46d7-9168-ee121fd0635b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.667920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.668229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.669504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.669689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.680785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 260/513] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:44.682386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.701166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.701835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.702123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.702399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.703149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.703455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.704892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.713012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 254/514] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:44.713794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.727699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.781904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d951-0d0c-4c7c-bc30-3a18dc91cdbc None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.811268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815905ae-8691-45f6-a60e-5498106c1ae6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.814240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.815032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.815311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.815578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] PATH_INFO: `/v3/users/glance` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.823086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.824730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:44.840166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:44.841248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 261/515] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:44.842268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:44.846233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.846902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.847185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.847462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.880194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.913211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.914705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:44.946662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74918a87-8307-4be7-a195-dacc70fcd01b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:44.953937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:44.982208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:44.982867 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:44.983151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:44.983427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:44.984242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:44.984553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:44.986002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.003911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 262/516] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:45.004707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.022412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 255/517] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:45.023263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.087455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.111284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc8fac-1c83-4c74-bffa-30c6f378e8d4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.122503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be456a-a270-42a4-90c3-876eb16a6346 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.148663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.149339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.149622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.149894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.150699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.152270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.156518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.157318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.157632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.157878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.168742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.170224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.172372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 263/518] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:43:45 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:43:45.173172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:45.192145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:45.193118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:45.194029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:45.194029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:45.194029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:45.194029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:45.194746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 256/519] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:45.195145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.199817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.286111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca0665-aeb7-491c-a078-820128da93fd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.307222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bfa16-4291-4e56-969d-4a3571a74cc1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.323566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.324308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.324596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.324905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.325734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:45.326050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.326439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.327692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.345956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.345956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.346245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.346450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.347295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:45.347607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.349113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.351515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 264/520] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:45.352381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.355020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 257/521] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:45.360866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.459863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.485805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11a04c-95e2-476b-9330-b962064f4c90 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.487722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d0b6d-fa5d-4331-a10a-5975a18ed158 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.534274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.534977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.535541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.535787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.536110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.536383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.536639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.537111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.537938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:45.538343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.539836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.549449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.551240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.558725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 258/522] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:45.558725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:45.578292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:45.579351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:45.580233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:45.580233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:45.580233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:45.580233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:45.582112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 265/523] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:45.583303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.600193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.659004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12759c5-0346-4e74-87a5-76ab22bab3d1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.694031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.694813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.695125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.695467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.703986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28365d2f-202e-4c8d-aa94-4d065c5a0d53 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.739251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.746219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.747453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.748039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.748529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.756848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.759153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.769184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.770647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:45.778063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:45.778958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:45.779859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:45.782234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 266/524] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:45.783309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.820271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:45.887911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 259/525] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:45.887911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:45.898003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124b2ce-5680-4099-9f50-a378f76eab71 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:45.939681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:45.943083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:45.943083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:45.943083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:45.943083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:45.943083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:45.945237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:45.957225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 267/526] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:45.958306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.009173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fd415-6b0c-4734-ac10-bb8c29ff7810 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.012977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.048330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.049304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.049804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.050241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.051404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.051904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.053592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.064062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbcbe6-0d4a-4e49-bef5-d314eb9185b5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.069141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.078570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 260/527] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:46.079761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.099624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.100575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.101091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.101194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.112391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.113975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:46.131775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:46.132907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:46.134325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:46.134325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:46.134325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:46.134325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:46.139819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 268/528] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:46.140819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.191243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a5553-f832-4c84-8340-45ccecec4a0b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.227638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.228728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.229144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.229549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.237034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.238805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.242208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d948ca-b224-46fb-ac27-43004b14541d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.244857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:46.254483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:46.255607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:46.256730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:46.260048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 261/529] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:46.261030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.280262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.281029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.281312 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.281587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.323804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.352968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.355526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.367691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc191-5ebe-4d03-8aa2-a3391eaae9de None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.402465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.403395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.403830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.404254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.405220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.405658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.407275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.420654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 262/530] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:46.421787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.466564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 269/531] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:46.467726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.471067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.531735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f2647-a1d5-4524-a943-a70a2593b19b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.557320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.571909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.572665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.572943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.573213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.574090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.574396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.575965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.581306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f4e5-eae6-4074-bcbc-d5a599b1bfb5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.595073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 263/532] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:46.595934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.617344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.618533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.619096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.619564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.627022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.628811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:46.644633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:46.645523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:46.646393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:46.649076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 270/533] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:46.650113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.699940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef53fd9-a002-4156-aba7-21e7db0173d9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.715203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.736516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.737524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.737956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.738362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.749306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.750984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.755412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d955ff0-8f35-43be-bdf6-8394fbc9a454 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:46.769923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:46.770942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:46.771983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:46.771983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:46.771983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:46.771983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:46.773507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 264/534] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:46.774554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.793213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.794031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.794506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.794870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.795816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=13be832163b9406583f63f7cd00651ae) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.796251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.798031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.804443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 271/535] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/role_assignments?user.id=13be832163b9406583f63f7cd00651ae&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:46.805175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.807832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.886986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf8856-e93d-418a-89e0-5684b48277ab None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.918037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c635e-caf0-4647-a834-0454f76417c9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:46.947451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.947451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.947451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.947451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.947914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.947914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.947914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:46.971235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:46.974424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 265/536] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:46.978855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:46.979546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:46.980610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:46.981275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:46.981551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:46.984784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:46.985098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:46.986539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:47.014894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 272/537] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:47.022359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.127034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:47.140626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d679e92-8682-413c-820b-bed59704e5a3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:47.172953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6b6df-137b-4b1f-8fa7-bd703948142e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:47.201584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:47.202253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.202534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.202812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.207896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=44cc172ac2904c84a84c7f4dbb707901) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:47.207896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:47.207896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:47.216013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 266/538] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:46 2022] GET /identity/v3/role_assignments?user.id=44cc172ac2904c84a84c7f4dbb707901&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:47.253732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:47.254908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.255390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.255891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.273505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:47.275031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-6ba03ddc-9818-4f47-8d3f-01f4e0850954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:47.302672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:47.303594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:47.304468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 273/539] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:47.310511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.448783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8205a-44e0-486d-b37d-d9e8bc37f444 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:47.484756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:47.485484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.485769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.486094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.486896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:47.487205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:47.488746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:47.495630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4ebf-a14e-4212-a391-f9d2bd2063f0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.495934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4ebf-a14e-4212-a391-f9d2bd2063f0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.496258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e4ebf-a14e-4212-a391-f9d2bd2063f0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.498157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 274/540] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:47.502657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 267/541] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:47.504398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8cfa-7f71-48f7-8eb6-3bbfd04047cb None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.504631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8cfa-7f71-48f7-8eb6-3bbfd04047cb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.504914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8cfa-7f71-48f7-8eb6-3bbfd04047cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.507954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.524014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5d5e8cfa-7f71-48f7-8eb6-3bbfd04047cb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:47.525346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5d5e8cfa-7f71-48f7-8eb6-3bbfd04047cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:47.527432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:47.576928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:47.608248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 275/542] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 10:43:47.609937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebad94-48ef-4a6e-8de2-5cb67ceb8d61 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.610230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebad94-48ef-4a6e-8de2-5cb67ceb8d61 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.610506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebad94-48ef-4a6e-8de2-5cb67ceb8d61 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.612247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 276/543] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:47.617141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.617426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.617765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.618368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181507-e645-4c2a-b15b-5899e02d21ba None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:47.646511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:47.647719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:47.658510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:47.659179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.659819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.660225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.661393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=69a3d1b0e64049d28adc1a4f9ea10337) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:47.661709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:47.663456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:47.671186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 268/544] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/role_assignments?user.id=69a3d1b0e64049d28adc1a4f9ea10337&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:47.672536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.772363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:47.800723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221ac4-c05e-4c4a-ae5e-7f7a20becc6b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:47.819789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 277/545] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:43:47.846640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.846971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:47.847739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:47.848045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:47.848321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:47.874308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:47.883115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:47.884461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:47.885643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:47.891361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 269/546] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:47.893100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:47.994560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe690d07-66fe-4773-823b-2f1383a3e786 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.014453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:48.063200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.064536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.066531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94b5ec-19a3-476e-8fee-43b677a1de4d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.067017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.067480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.089001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:48.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:48.103985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:48.104781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:48.110232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 278/547] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:48.120675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.138106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.138808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.139090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.139353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.140176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:48.140453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.141905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.170302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 270/548] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:48.176903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.183124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:48.239969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df6c03-5c74-4b35-8231-abb8e9e2af20 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.275892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.276690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.276879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.277119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.277852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:48.278157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.279541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.292206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 279/549] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:48.293658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.332551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:48.391394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac720d2-e79c-47a1-987b-d90ca559ef0e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.431014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077aff-d41b-43bf-ac22-5134b946afd2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.434185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.434987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.435262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.435530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.455937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.457411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:48.488446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:48.489860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:48.490875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:48.490875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:48.490875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:48.490875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:48.490875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 280/550] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:48.501432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.502250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.502530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.502806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.503793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.526774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.528290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:48.549918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:48.550833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:48.551700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:48.551700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:48.551700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:48.551700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:48.551700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 271/551] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:48.556715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.634730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:48.656359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f4c5b-42f1-4942-9da3-29428155f98c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.690581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.691252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.691528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.692038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:48.692455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.693185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:48.693488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.694896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.720933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 281/552] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:48.726091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.768474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3056ee-0d32-4653-a3a1-b8eb67e8a1e5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.838123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.838798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.839077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.839347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.840105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:48.840413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.841815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.866986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 272/553] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:48.873010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:48.891581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edd231-4929-401f-a260-ef62730c705c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:48.947742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:48.948855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:48.952278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:48.952756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:48.965686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:48.969710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:48.993469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:48.994585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:48.995959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:49.007708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 282/554] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:49.009892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.032260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.071631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04894c58-5d59-4e0e-9fbb-00a5ac8be3a7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.096580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.134478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.135326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.135601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.135888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] PATH_INFO: `/v3/users/swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.150905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.152315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:49.175499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:49.176536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:49.178707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 273/555] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:49.188865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.191789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2d193-453c-4c0e-9967-08a77c2c6eb0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.226847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.227533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.227815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.228125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.229304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:49.229619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.231099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3116c-7787-4ff1-be16-d176aefb2b25 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.249658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 283/556] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:49.251080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae554109-2227-4b8e-a5ed-c8d49dc144db None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.251368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae554109-2227-4b8e-a5ed-c8d49dc144db None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.251639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae554109-2227-4b8e-a5ed-c8d49dc144db None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.255913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 284/557] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:49.255913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae554109-2227-4b8e-a5ed-c8d49dc144db None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.325284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f6c15-7097-4e36-ad20-7d4d5cdcf657 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.338924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.362805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.363704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.364193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.364679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.365710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:49.366212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.367837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd63c23-6f35-4293-bb7d-95ea35b5088c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.391982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 274/558] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:49.393675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.394226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.394716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.395640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae554109-2227-4b8e-a5ed-c8d49dc144db None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.411736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:49.413192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:49.439706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.440693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.441430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.442260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.442260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.457553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.459190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:49.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:49.487079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:49.488001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:49.488001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:49.488001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:49.488001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:49.488551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 285/559] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:49.489503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.490904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 275/560] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:49.499920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.630155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec24ae-9166-4252-8e52-e80cb2d4ebba None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.637875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555098f-d529-41d2-bc32-eaa0b2fc1b10 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.645754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.675373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.675987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.676338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.676625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.677349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:49.677653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.679260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.681090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.681986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.682301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.682622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.698000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 276/561] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:49.699030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.738636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.790609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:49.792324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:49.819897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289049-1bef-41dc-a9dd-4b81eeb49be4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:49.855081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:49.856033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.856315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.856588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.867356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-83403340-303d-42d5-9dfe-d8ae761fdfc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:49.875368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:49.876486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:49.877352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 277/562] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:49.878250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837f91-372e-4334-9815-5a41cfb9e741 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:49.878531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837f91-372e-4334-9815-5a41cfb9e741 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:49.878802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837f91-372e-4334-9815-5a41cfb9e741 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:49.884855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 278/563] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:49.885615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd837f91-372e-4334-9815-5a41cfb9e741 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.933948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 286/564] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:49.934121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:49.953858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:49.992204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.019902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd837f91-372e-4334-9815-5a41cfb9e741 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.053394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.054131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.054410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.054683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.055332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff71e-11b5-4e00-af64-5dc42fb80cf9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.061511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.062938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d44708d0-8732-4e8e-8c32-81408f23037e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:50.077059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:50.083187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 279/565] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317357-28c8-4f5e-af9d-38a71b10b45a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.084142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317357-28c8-4f5e-af9d-38a71b10b45a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.084758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317357-28c8-4f5e-af9d-38a71b10b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.090084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.090785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.091051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.091327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.092104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:50.092413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.093795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.096310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-94317357-28c8-4f5e-af9d-38a71b10b45a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:50.097491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-94317357-28c8-4f5e-af9d-38a71b10b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:50.101903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 287/566] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:50.102684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.162718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 280/567] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:50.164170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3b8cd-8d3c-448e-b19a-7067d6dc9246 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.164517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3b8cd-8d3c-448e-b19a-7067d6dc9246 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.164953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3b8cd-8d3c-448e-b19a-7067d6dc9246 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.166458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 281/568] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:50.167160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3b8cd-8d3c-448e-b19a-7067d6dc9246 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.186494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.211536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0bf865-0e58-49b8-aecb-f82824a86fc3 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.239794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.245460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.246347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.246707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.247053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.258072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.259972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.270281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3b8cd-8d3c-448e-b19a-7067d6dc9246 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:50.277648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:50.278701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 288/569] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:50.284562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.303891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.304553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.304827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.305098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.305877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:50.306204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.307867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ceae-27d9-43e3-ab43-11acf3c1f1cc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.319287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 282/570] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:50.320896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.321021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.321245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.337407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:50.338608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:50.384211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209e16d-fb13-44c8-a7ea-97594a41bd2c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.405253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 283/571] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:50.406090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.417462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.418265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.418549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.418826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.420377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.431251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.432803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:50.449879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:50.450947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 289/572] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:50.451998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.472973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.510922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf008a-6818-468a-a35f-56f9a54d06d4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.543642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.544473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.544752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.545024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.551495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23263a11-8761-42c0-9b7b-bdfb9f45ee7c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.555089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:50.563003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:50.563996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 284/573] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:50.565187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.587410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.588272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.588565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.588880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.589591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:50.589897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.591290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.602875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 290/574] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:50.603706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.646061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.665941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c139bb-71db-4872-94b7-75704aa4daba None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.687668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.699348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.700337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.700553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.700820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.709213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63094c83-17ad-46b0-b0d3-97ba03b4cab9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.715740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=d08cdb2d283143aa83b38bb3b04aee8c, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.717408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.732157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 285/575] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:50 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:50.733037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.744183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.745599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.745896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.746157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.759441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:50.768231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:50.769279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:50.770114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 291/576] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:50.771274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.857984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2c5f3-df96-4b58-ac41-3bb9201e35a0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.892179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:50.895695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee563bcb-5b17-42c8-8a2e-46e09cdffc85 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:50.900626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.901343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.901590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.901860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.902579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:50.902917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.904757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.919785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 286/577] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:50.920785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.940026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:50.940687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:50.941034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:50.941379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:50.942210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:50.942613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:50.944110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:50.953404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 292/578] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:50.954227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:50.967554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.055935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398bf2-cf6e-4e8b-af89-a09821d6ba16 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.070398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bafdc2-29a1-4593-88db-e66189c092f0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.090681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.091567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.091940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.092277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.099276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.100821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.106672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.107421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.107710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.108015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.108774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.109085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.110459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:51.115159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:51.116275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:51.117187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:51.120025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 287/579] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:51.120905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.128160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 293/580] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.129223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.158026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.222262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.234172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247065de-71fb-4136-a676-a6f6e28a6ee6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.240287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21673b03-4825-42aa-9af3-83dfc7149343 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.272933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.273907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.274192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.274535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.279992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.281304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.281798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.282270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.283340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.284899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.296806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.298282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:51.302388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:51.303345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 288/581] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:51.310209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:51.320699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:51.321809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:51.322973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:51.322973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:51.322973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:51.322973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:51.328637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 294/582] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:51.328637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.403820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.424846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1515-1978-4bc6-8c26-7abb1c9f7e7f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.430809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f27b77-07f8-4f31-bb05-06f6dc2cb44e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.462746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.463483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.463761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.464068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.464876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.465175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.466641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.468846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.470086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.470466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.470839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.483065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 289/583] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.483065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.488479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.490448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.494792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:51.511183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:51.512233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:51.513109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:51.513109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:51.513109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:51.513109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:51.513109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 295/584] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:51.517662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.585692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58db34-36ba-4a00-b026-3b1167208007 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.619284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.620210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.620493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.620773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.621102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f39622-4930-453a-ace6-ceb111328d37 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.621607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.621919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.623358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.641724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 290/585] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.642551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.652533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.666170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.667153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.667633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.668143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.669745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.670263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.673042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.693178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 296/586] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.694169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.716482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.751321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ffbe0-1a57-46e3-888b-8900f3bcb761 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.788387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.789147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.789439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.789714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.790644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=6f958e0eb8444871851792cb254b56a7) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.791056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.792635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.798420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 291/587] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/role_assignments?user.id=6f958e0eb8444871851792cb254b56a7&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.798830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e176b-5e2e-46ad-8e20-31b43a9af016 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.799196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.858548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.859751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.860069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.860385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.861845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:51.862168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:51.863961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:51.882756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 297/588] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:51.884285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:51.935654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:51.948412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d0df-c6df-4183-838e-ba2f0f09a860 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:51.994920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:51.995747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:51.996074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:51.996349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:51.998485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.007863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.009702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.011650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efaa7b-0579-4439-a577-438546a7e658 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:52.027820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.028958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.030066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:52.030066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:52.030066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.030066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:52.030066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 292/589] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.030347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.050271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.051403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.051920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.052327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] PATH_INFO: `/v3/users/glance` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.062209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.064010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:52.079075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.080668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:52.081755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:52.088433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 298/590] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.089619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.130750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3cb0-74ff-4f02-821a-a71318d196c1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.167722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.168617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.168903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.169179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.176864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.178346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:52.193713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.194691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:52.195613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 293/591] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.196234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.202070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80174c5f-d0bf-49c6-a870-e6b08497d7da None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.212244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.238407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.239168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.239590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.239957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.240755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.241219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.242697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.244791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.263162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 299/592] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.264324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.305442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0459-1e42-4927-8495-cdf38b962411 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.344310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.345098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.345381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.345659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.346509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.346931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.348402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.368959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 294/593] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.369751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.372509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91d37b-6c6a-474d-a8f4-c750516a8998 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.415834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.416618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.416904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.417471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.418609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.419046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.420781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.447400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 300/594] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.447400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.475281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.479135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741dfad-8526-4eae-8cde-5dc48bec2901 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.482125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.516141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.517189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.517621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.518031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.525282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.527024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:52.541876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.543009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:52.544130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:52.550915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 295/595] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.552039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.561561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a086c0-b745-409f-a3bf-ea338258a449 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.604397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.605688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.606032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.606369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.619957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.621705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:52.645224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.646368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.647451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:52.647451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:52.647451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.647451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:52.652019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 301/596] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.652989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.667493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117b38f-126b-42eb-9fad-9430391c2bb6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.703431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.705327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.705997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.706744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.719763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.722660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.738703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:52.743387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:52.745072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:52.745958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:52.745958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:52.745958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:52.745958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:52.752172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 296/597] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:52.753313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.763813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c631000-546e-403e-89f0-ad7df692a46e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.775172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.798030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.798691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.798968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.799237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.800048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.800356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.801797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.813386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 302/598] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.814504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.862836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9ec1b-85dd-4738-b9a1-0f0386be58f3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.899332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.900039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.900512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.900858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.901684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.901989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.903395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30fca5-ce2f-46c2-9563-5ed173aac33f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.919935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 297/599] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.925698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e06a7-7f8b-4660-8833-4facd22af104 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.925982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e06a7-7f8b-4660-8833-4facd22af104 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.926081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b9d5-0c6e-455e-a17f-90707aea23cb None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:52.926252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e06a7-7f8b-4660-8833-4facd22af104 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.927537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 298/600] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:52.931800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e06a7-7f8b-4660-8833-4facd22af104 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:52.967209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:52.968152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:52.968426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:52.968700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:52.969424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:52.969724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:52.971111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:52.977764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:52.992035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 303/601] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:52.992952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.019417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.049011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e06a7-7f8b-4660-8833-4facd22af104 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.086396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.087290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.087571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.087947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/13be832163b9406583f63f7cd00651ae/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.097878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f8ca0-5714-441d-9356-6219eeacdc5b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.103602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=13be832163b9406583f63f7cd00651ae, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.105187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75789c41-df29-4c19-9dfa-55a6cc020651 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.120024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 299/602] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:52 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/13be832163b9406583f63f7cd00651ae/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:53.121358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.121708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.122052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.141864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:53.143328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:53.149097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.150225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.150759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.151452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.162349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.164355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:53.185144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:53.186377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:53.187456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:53.191030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 304/603] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:53.192198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.226807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 300/604] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:53.228019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.275129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.297093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.330167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9226eb9-c67d-442f-9863-b9b3f7852aae None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.351265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040b574-c519-4921-92a2-29b294d329a4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.367631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.368472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.369039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.369213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.376655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.378226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:53.394108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:53.395050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:53.396003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 305/605] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:53.397000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.401165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.402126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.402633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.403123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.404146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:53.404656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.406971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.424342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 301/606] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:53.425589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.510617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a894a-eda1-4829-b7ab-835dec598f8f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.529150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.550084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.551027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.551410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.551786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.563385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077617e-a639-4210-81df-2d64cc259433 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.563759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:53.572490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:53.573696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:53.574885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:53.581698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 306/607] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:53.582570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.584446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.607772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.608758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.609191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.610135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.611360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:53.611828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.614161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.632077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 302/608] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:53.633166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.694410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5da74-04b0-4b6b-b7d2-97af3d17ceee None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.735375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.736510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.736896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.737313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/44cc172ac2904c84a84c7f4dbb707901/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.756937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=44cc172ac2904c84a84c7f4dbb707901, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.759367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.767499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa95a7b-1e5e-41d5-92ba-b64d55b6c2e8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.784603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 307/609] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:53 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/44cc172ac2904c84a84c7f4dbb707901/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:53.786455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.813672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.814235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.814521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.814796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.815608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:53.816096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.817591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:53.830761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.836010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 303/610] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:53.838991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:53.895002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:53.927893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418aa97-2b96-43cb-b784-cd72eab3a9e9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.969883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:53.971291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:53.971770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:53.972170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/69a3d1b0e64049d28adc1a4f9ea10337/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:53.976392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7313c3d-b50b-4082-9b2d-5d4eaaa3fb11 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:53.992420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=69a3d1b0e64049d28adc1a4f9ea10337, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:53.994815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad639a-d73c-4549-887e-854c74dc482c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.022136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 308/611] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:53 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/69a3d1b0e64049d28adc1a4f9ea10337/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:54.023372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.024538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.025166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.025698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.039669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:54.041719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.043041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4371c-18bf-4337-81b5-3c89a17f4943 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.043335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4371c-18bf-4337-81b5-3c89a17f4943 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.043617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4371c-18bf-4337-81b5-3c89a17f4943 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.046243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 309/612] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:43:54.055947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.056394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.056827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:54.064932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:54.066065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:54.067198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:54.067198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:54.067198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:54.067198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:54.069181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 304/613] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:54.074780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.085893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:54.087542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:54.174439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 310/614] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:54.183452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:54.193520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5309e-2ccf-45fe-84a3-4766e4b0843f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.215466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.232973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.233336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.233610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.233889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.234631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:54.234940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:54.236251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:54.236388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.250058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 305/615] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:54.255409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.353275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9eddb-bc51-4ba0-a213-016056ef8ee6 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.362711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a2975-ff85-415c-b9e2-54a6c231ef6c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.391530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.392388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.392665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.392921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.400974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.401780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.402063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.402438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.405160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:54.410411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:54.411901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:54.413197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:54.414096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:54.415375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 311/616] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:54.422404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:54.427149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:54.428232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:54.429628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 306/617] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:54.434573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.447495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:54.529747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:54.597042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91352e5-5c23-437e-8590-dbe4a8776c21 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.616276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8a8e6-c4bc-4c68-9322-e6b921279e0e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.671081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.671773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.672081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.672361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.673216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:54.673531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:54.675028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.711596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.712994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.712994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.713290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.714282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:54.715906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:54.716017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:54.717297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 307/618] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:54.722890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.739547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 312/619] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:54.750084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:54.835668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:54.895144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb121a8-2b38-4199-877d-41566a8423f4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:54.969867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:54.970543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:54.970820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:54.971118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:54.976415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7989-77db-4a13-b23d-00f7f864f05f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.031081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.052003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.052831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.053111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.053382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.077018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.082358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.106920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.108432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:55.123581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:55.124753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:55.125653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:55.125653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:55.125653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:55.125653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:55.132270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 313/620] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:55.136906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.178583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.217130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 308/621] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:55.222603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.262630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea822-a4e7-4ed5-a467-35195853db09 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.303453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.304573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.305087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.305563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.306489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:55.307058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.308799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.326982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 314/622] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:55.328652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3838433-cc60-4d41-9859-9ed52399cdaa None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.328771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.376611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.377439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.377720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.377993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.382625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.395706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.397214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:55.427693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:55.428772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:55.430112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:55.430112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:55.430112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:55.430112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:55.430112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 309/623] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:43:55.434786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.454144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ca1a6-7ef7-4583-a5e8-857091a48244 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.472514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.492227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.493193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.493577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.493959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.501835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.507149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-ba898d9b-b38a-4f03-b76b-43d24f1b20e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:55.522581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:55.523783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:55.524883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 315/624] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:55.525644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b483f4-aa8a-47ae-ab77-be0790274e0f None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.525766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b483f4-aa8a-47ae-ab77-be0790274e0f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.526061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b483f4-aa8a-47ae-ab77-be0790274e0f None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.531414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 316/625] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:55.532231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b483f4-aa8a-47ae-ab77-be0790274e0f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.585797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33d007-d5ea-4503-9856-821e9fb36cd0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.623138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.624097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.624936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.625828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:55.626274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.628073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655d4bc-eaa0-4f51-9711-a795275bde63 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.636602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.641185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b483f4-aa8a-47ae-ab77-be0790274e0f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.646497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 310/626] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:43:55.646658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.647002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.647138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.670975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:55.672197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:55.687695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.688832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.689258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.689913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.691296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:55.691776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.693891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.717102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 317/627] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:55.718231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.763157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 311/628] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:55.764134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.777149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.849574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76447db-bee9-4488-93f9-2791bd6c30ef None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.885818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.886621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.886921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.887161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.894014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.895585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.898934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d08e-9afe-483c-b149-a6943f9ba292 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:55.909695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:55.910649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 318/629] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:55.912573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:55.933687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:55.957517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:55.958481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:55.958872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:55.959268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:55.974355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:55.976204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:55.999162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:56.000352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:56.001460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:56.001460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:56.001460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:56.001460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:56.004025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 312/630] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:56.004988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.041273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86da90-93a5-460f-97d8-b73a6ec3b5f5 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.073441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.093921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.094955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.095478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.095827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.112087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:56.117169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2453-fcd9-4939-824e-40339bff4c07 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:56.122456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:56.123437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:56.124342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 319/631] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:56.125055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.175916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.176603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.176889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.177166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.177972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:56.178284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.180414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.198863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 313/632] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:56.200215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.220361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.248031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd42e7-638f-4456-97db-d709c2acb6b0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.296129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.299498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.299786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.300121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.300861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:56.301170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.303115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.324555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 320/633] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:56.324555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.334605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b675b80-e0f3-4625-84a9-4f5d5e5f759c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.358486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.372886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.373946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.374429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.374930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.375932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:56.376438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.378016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.392155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 314/634] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:56.393352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.445050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76029ffb-2bf9-4552-8be5-1578a09f0cdd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.481722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.482294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.482575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.482853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.483686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:56.484032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.485487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.491938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 321/635] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:56.492805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.495579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.505346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd649d4-1a67-4808-b164-504dc01ccd41 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.546590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.547404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.547679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.547978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.555772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.557512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f8397d1c-0322-4279-9729-204adf28679e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:56.573723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:56.574719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:56.575614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 315/636] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:56.576612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722e68-617f-4e45-a60f-6bd43558b5f3 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.576612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722e68-617f-4e45-a60f-6bd43558b5f3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.576877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722e68-617f-4e45-a60f-6bd43558b5f3 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.578356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 316/637] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:56.579332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6b19-b489-4304-806e-483466d37b02 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.579606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6b19-b489-4304-806e-483466d37b02 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.579898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d6b19-b489-4304-806e-483466d37b02 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.581304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 317/638] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:56.585754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.586183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.586565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.604853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:56.606269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:56.624631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eccbcf-f73f-4095-9f48-acfe167dc427 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.650180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.662974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.664008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.664102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.664344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.677290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.678769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.682160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 318/639] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:56.683168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-24efb8ba-f202-4a2b-a418-39ee9a6891ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:56.698107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:56.699163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:56.700153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:56.700153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:56.700153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:56.700153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:56.700375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 322/640] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:56.701499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.701784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.702065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.719230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:56.720468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:56.778134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.797735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 323/641] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:56.798577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.799130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181af3b-3f0e-4558-99e3-ba7d695cd8c3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.843693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.844413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.844686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.844973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.845741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:56.846044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:56.847502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:56.864939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 319/642] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:56.864939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.905298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:56.919083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae5635-30c5-4771-93df-4a919f67b209 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:56.955802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:56.956803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:56.956936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:56.957165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:56.967784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:56.975388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:56.976412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:56.977258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 324/643] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:56.977962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:56.988448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982e93c-5c99-4977-9781-03d81d418875 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.023522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.024223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.024508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.024784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.025571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:57.025914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.027256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.029276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.040745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 320/644] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:57.041837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.094273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5738-0939-43f4-8bbd-3662c5ebc073 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.130540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.131344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.131622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.131912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.141889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.142621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:57.149868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7fc9-c76f-4384-b388-a032448846a7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:57.150981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:57.151923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:43:57.152936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:57.157123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 325/645] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:57.157758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.190419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.191320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.191601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.192033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/6f958e0eb8444871851792cb254b56a7/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.209485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=6f958e0eb8444871851792cb254b56a7, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.210955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.226405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 321/646] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:43:57 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/6f958e0eb8444871851792cb254b56a7/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:43:57.227008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.287495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d045-b18e-478e-85f2-3aec7d4fec4d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.293084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.345721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.347517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.348012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.348456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.349372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:57.349824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.352381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.364469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8b2a-6c60-42ac-b68d-b612df2ba4d9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.366863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 326/647] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:57.368823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.408733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.409541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.409818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.410085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.418813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.420538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:57.438491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:57.439606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:57.440734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 322/648] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:57.441328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.445405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.501156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c708-ad67-4709-b80c-a20cd642ed08 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.551116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.552279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.552724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.553154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.554162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:57.554629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.556330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.566527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 327/649] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:57.567680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.579026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918a7fd-988e-4426-8d1a-9b1cb1308f1c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.601455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.625258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.626314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.626593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.626865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.643073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.644626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:57.667261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:57.668463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:57.669646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:57.669646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:57.669646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:57.669646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:57.669646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 323/650] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:57.669949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.699909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822466-6732-45ac-86bb-45e4d3ec788e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.743919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.745421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.746287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.746688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.747063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.748054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:57.748466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.750014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.773429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 328/651] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:57.774477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.798252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2a74-5dea-49d2-b540-2823bd6118df None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.834984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.835941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.836331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.836706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.848112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.849692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:57.867701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:57.868805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:57.869888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:57.869888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:57.869888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:57.869888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:57.875898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 324/652] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:57.876835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.881476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:57.901880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e5d3-c187-4c55-8d8e-94320470a4c1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:57.946311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:57.947478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:57.947971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:57.948625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:57.949892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:57.950616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:57.952774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:57.967926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 329/653] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:57.969380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:57.993329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c106a-87ab-4fd4-89fb-2e916f0cd61c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.004258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.044637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.045989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.046761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.047658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.100179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c2fed-620d-4fb0-a235-f253323de2e2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.121977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.123774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.145061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.146664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.147200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.147960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.149197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:58.150126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.152476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.169567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 330/654] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:58.170715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.180323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.246135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 325/655] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:58.251161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.262769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.287838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809817b-d769-404b-9afb-bff3d320a858 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.323475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.324325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.324601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.324871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] PATH_INFO: `/v3/users/glance` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.333332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.334806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:58.350360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:58.351778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:58.352785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 331/656] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:58.353422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.383727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7405a-81d1-4f55-9718-be7fe07fc16d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.423858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.424535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.424808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.425082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.432864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.434318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.444414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:43:58.450502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:58.451485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:43:58.452475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:58.456135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 326/657] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:58.456976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.492964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cccb0-cbab-4875-b570-1dcd9548ae7d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.533173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.534184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.535124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.535462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.535800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.546827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.548455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:58.566619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:58.567582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:58.568520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:58.568520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:58.568520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:58.568520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:58.572856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 332/658] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:58.573785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.577270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7db3a-206c-4eb9-81b9-30a2239e7c91 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.621804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.623061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.623526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.624126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.625467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:58.625969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.627599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.647681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 327/659] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:58.648836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.676506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779594d-1c08-4bac-aaaa-e7f4cb8e9c4e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.693486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.712582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.713326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.713607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.713913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.714744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:58.715052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.716599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.737235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 333/660] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:58.738280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.761973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2a97-09a3-4d11-941e-b5c4f7bf3663 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.789106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.800012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.800684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.800955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.801229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.801953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:58.802256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.803696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c5c1c-c164-4f0e-8d6f-172a92f256b5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.815743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 328/661] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:58.817269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbde982-1c6c-45e0-aca8-0527dcaa7f5b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.817738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbde982-1c6c-45e0-aca8-0527dcaa7f5b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.818196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbde982-1c6c-45e0-aca8-0527dcaa7f5b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.819845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 329/662] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:58.820980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbde982-1c6c-45e0-aca8-0527dcaa7f5b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:58.851216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43d99-6c5b-44ab-95af-7116038e139e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.887185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.887748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.888199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.888489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:58.929808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbde982-1c6c-45e0-aca8-0527dcaa7f5b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:58.939293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:58.965807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:58.967489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:58.967656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:58.968311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:58.968595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:58.968871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.027336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.043203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.044800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7287b-367b-4f07-a710-26bea7181dec None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.079153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 334/663] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:59.080051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.156740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.159227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 330/664] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:43:59.160603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.160893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.161189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.178900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:59.180132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:59.187411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6973-bc75-41a9-af9e-c90551b8c363 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.228391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.229616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.230056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.230331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.237514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.238986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:43:59.252833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:59.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:43:59.254641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:59.258769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 335/665] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:59.260058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.260304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 331/666] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:59.261180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.283135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.363688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ac28d-3cb6-4c09-9582-95837073d665 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.371240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d387c96-45c2-4cda-91a4-b49950a46241 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.397146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.397951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.398233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.398512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.401215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.405487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.406376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.406658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.406981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.409658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.411136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.417314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.418836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:59.427809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:59.428831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:59.429752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:59.429752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:59.429752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:59.429752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:43:59.429752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 336/667] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:59.430031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:43:59.436387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:59.437447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 332/668] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:59.438349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.505154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.530227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbb878-2994-4f97-b1be-7e0d1da2df79 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.537369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c328c6e-59a3-4f33-968e-4c436420d589 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.562487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.563137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.563414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.563684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.564506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:59.564813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.566285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757e7a0-c3b8-4465-a930-3e33c671f58a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.570751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.571537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.571815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.572115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.578180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 337/669] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:59.579409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd1fb-98e7-4a1c-aab2-f853d918686b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.579695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd1fb-98e7-4a1c-aab2-f853d918686b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.580000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd1fb-98e7-4a1c-aab2-f853d918686b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.581468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 338/670] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:43:59.581989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fd1fb-98e7-4a1c-aab2-f853d918686b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.584150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:43:59.592228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:43:59.593247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:43:59.594095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 333/671] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:43:59.594809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.636712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.687622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fd1fb-98e7-4a1c-aab2-f853d918686b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.693460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd7587-23cf-429b-ad12-719e5c5a8c90 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.719152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.720915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.721587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.721925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.722205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.722925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:59.723243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.724656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.727593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.728276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.728553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.728856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.729617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:59.729911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.731308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ca18f-1ca6-4db5-b719-1206dc250b4e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.736244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 339/672] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:59.737208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.745822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 334/673] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:59.747421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.747710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.748019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.764758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:43:59.765971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:43:59.834226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 335/674] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:43:59.835023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.843570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f06c0-fe85-40dd-a625-d4b332f636db None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.852778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:43:59.877925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:43:59.878580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:43:59.878856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:43:59.879131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:43:59.879980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=13be832163b9406583f63f7cd00651ae) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:43:59.880298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:43:59.881736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:43:59.888481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 340/675] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/role_assignments?user.id=13be832163b9406583f63f7cd00651ae&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:43:59.889257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:43:59.962743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f735b0-a009-43a9-b331-d84ee1c010e3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:43:59.983711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.004157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.005201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.005610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.006064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.007077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.007590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.009824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.011074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9fa3-e0ef-4add-907a-be91ed600a25 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.025691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 336/676] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.025691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.055724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.058282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.058426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.058708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.067184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.068815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:00.084309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:00.085245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 341/677] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:00.086328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.119291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.143557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa9a6-c132-4c72-b769-1da9852f2d40 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.180039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.181287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.181770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.182259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.189287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffb774-2b39-4350-8490-ed8b973ffe2d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.190203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.192116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:00.207970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:00.208891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:00.209802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:00.210728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 337/678] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:00.211423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.224957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.225762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.226051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.226328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.237085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:00.244863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:00.245787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:00.246664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 342/679] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:00.247374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.265157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.328562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979403e8-72ee-43ab-8bcf-2225deca4200 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.356652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.365382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5746c79-5e7d-479d-937b-d548f9dfa74c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.368899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.369802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.370123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.370404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.381922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.383397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.401465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.402130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.402411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.402670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:00.403377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:00.404473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:00.412117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:00.412117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:00.412117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:00.412117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:00.412388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.412388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.412552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 338/680] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:00.412640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.412722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.431504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 343/681] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.431504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.514529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.524197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff475f-c292-45d1-af6f-ca72269f41fc None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.550125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466320d-79dc-44b1-8671-45746fc7a794 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.559484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.560252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.560546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.560821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.561599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.561923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.563354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.575557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 339/682] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.576440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.591735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.592787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.593140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.593487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.594308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.594705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.596116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.604848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 344/683] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.605672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.622369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.691430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8b5f-8b22-42d2-a118-1002b0d57058 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.705854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616321f0-dbc2-43cf-b1fe-f2fd316228fc None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.728385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.729150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.729433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.729708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.730434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.730741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.732161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.743256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.744156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.744577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 340/684] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.744860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.745346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.745459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.746459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=44cc172ac2904c84a84c7f4dbb707901) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.746776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.748291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.754874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 345/685] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/role_assignments?user.id=44cc172ac2904c84a84c7f4dbb707901&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.755663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.771492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.851500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:00.870926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda2f5-75a9-4989-ba8d-8149c0006341 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.887108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ac2c1-eca6-4ecb-970d-8c457ea16516 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:00.911976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.912658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.915914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.916189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.917329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=69a3d1b0e64049d28adc1a4f9ea10337) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:00.917643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.919469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:00.928255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 341/686] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/role_assignments?user.id=69a3d1b0e64049d28adc1a4f9ea10337&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:00.929902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:00.944827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39530996-2579-4976-b691-be24725de698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:00.945985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39530996-2579-4976-b691-be24725de698 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:00.947341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39530996-2579-4976-b691-be24725de698 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:00.947964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39530996-2579-4976-b691-be24725de698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:00.973829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39530996-2579-4976-b691-be24725de698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:00.975610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39530996-2579-4976-b691-be24725de698 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-39530996-2579-4976-b691-be24725de698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:01.000772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:01.001905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:01.002801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:01.002801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:01.002801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:01.002801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:01.003354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 346/687] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:01.008717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39530996-2579-4976-b691-be24725de698 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.052966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.098657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf25f-231d-4e93-afab-e26a3dba8b1b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.147389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.148202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.148502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.148775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.151428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39530996-2579-4976-b691-be24725de698 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.157327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.158797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:01.175958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:01.176969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:01.177925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 342/688] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:01.186937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.187441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.196252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.196926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.197210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.197852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.198950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:01.199277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.200819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.215842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 347/689] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:01.222345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.332620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5691-b2bb-4877-a33c-7f9800b810e0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.355433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3d42a-ca4d-4e19-bbb8-0cf00e5d27bc None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.394633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.395446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.396314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.396605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] PATH_INFO: `/v3/users/swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.405577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.407073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:01.424423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:01.425413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:01.426439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 348/690] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:01.433669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.443188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.443897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.444245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.444520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.445373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:01.445688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.451057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.453202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.483681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 343/691] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:01.489224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.498054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.558294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200e613-1ffb-4c45-8ca1-f93b71d4f596 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.601386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.602316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.602743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.603146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.604528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:01.605349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.606957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.627833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 349/692] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:01.636649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:01.750559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdebed-328c-4dc7-818e-f72dbc0a08e4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.752748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397db5-fe14-4cb7-ab00-48b7cced553c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:01.765245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.800662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.802323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.805559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.806785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.831382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:01.832394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:01.833663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:01.834636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:01.949267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.951487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:01.960244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:01.972244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:01.974786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493a14d-debd-422f-b54b-c2f105452ab4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.102926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 344/693] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:44:02.104514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7544330-5adb-4b46-b40c-feee4e6a40df None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.104947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7544330-5adb-4b46-b40c-feee4e6a40df None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.105264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7544330-5adb-4b46-b40c-feee4e6a40df None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.105436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 350/694] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:44:02.107924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 345/695] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:02.109078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.112688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.113207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.113682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.124498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.137102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:02.138820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:02.224332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d6a0-c790-4ca0-9b52-cdf3421d815c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.227953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 346/696] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:02.229178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.270419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.274813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.276310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.276960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.277447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.290957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.292944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:02.311265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:02.312492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:02.313428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:02.313428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:02.313428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:02.313428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:02.315545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 351/697] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:02.316584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.367937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75073c79-0258-4ad2-bd86-4c7a9c07e898 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.387774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.407040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.408291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.408801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.409291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.420536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.422298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.438251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78a0c-efae-418b-b812-3167b0553d49 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:02.441683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:02.442565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:02.443842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:02.443842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:02.443842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:02.443842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:02.446215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 347/698] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:02.447274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.480281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.481279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.481566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.481842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.482491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.483917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:02.502258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:02.503262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:02.504175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:02.507951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 352/699] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:02.508974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.542601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.574350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998d97b-4353-4a6b-b5f3-42a7b46e186f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.616510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.617439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.618096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.618448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.619581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:02.619974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.621391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e49b-1deb-4962-b3eb-1985fcc3a8aa None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.621506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.635996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 348/700] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:02.637398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.648462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.661629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.662386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.662734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.663083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.664300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:02.664680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.666500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.682112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 353/701] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:02.683368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.750970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db20dd-9b19-4f3a-b153-798f798f5d6b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.783686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0ef6-186c-4646-a829-804ac1074e2e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.789940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.790895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.791485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.792120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.793038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:02.793564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.795313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.797750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.802321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 349/702] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:02.803392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.825124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.825967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.826246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.826519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.833602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.835063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:02.849468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:02.850420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:02.851323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 354/703] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:02.852342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:02.898260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:02.932611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc400dc-80f7-46dc-a01c-5e003680cd4e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.953612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8857b7-9503-4299-85ce-638336b40f42 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:02.977437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.978659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.979304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.979915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.987810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.988287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:02.988950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:02.989226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:02.989495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:02.990201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:02.990503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:02.990933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.992075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:02.998363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 355/704] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:03.004959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:03.007902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:03.008814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:03.010084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:03.012819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 350/705] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:03.018983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.155840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.177593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8f52-58fe-43eb-8ace-29b02ce0fdd2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.195798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.207007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d846c44-dc34-470b-be3d-75de507c8108 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.226743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.227578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.227723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.227971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.228761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:03.229174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.230767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.245086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 356/706] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:03.247296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.248244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.248747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.249233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.250251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:03.250360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.250858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.252539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957529-2d53-4ca4-998a-d5d67e065cc8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.267316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 351/707] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:03.269068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b769179-6b4e-4a9a-8043-2c107b5cb5f2 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.269617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b769179-6b4e-4a9a-8043-2c107b5cb5f2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.270206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b769179-6b4e-4a9a-8043-2c107b5cb5f2 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.271936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 352/708] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:03.272941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b769179-6b4e-4a9a-8043-2c107b5cb5f2 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.357991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739c9e1-8deb-489a-979b-b78c2ed1e8d8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.389245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b769179-6b4e-4a9a-8043-2c107b5cb5f2 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.398209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.399472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.399985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.400838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.402319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:03.402994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.404998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bf095-ffc2-4a97-87a9-85ba582a1d22 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.416578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 357/709] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:03.416578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.416578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.416578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.425277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.434455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.435203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.435492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.435768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.436761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=6f958e0eb8444871851792cb254b56a7) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:03.437070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.437677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:03.438547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563633c-9ae1-4401-a8b8-d7fa9109f1de None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.438891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:03.446239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 353/710] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/role_assignments?user.id=6f958e0eb8444871851792cb254b56a7&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:03.447733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987b56-5ed4-4455-952e-5cff68286a6f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.448164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987b56-5ed4-4455-952e-5cff68286a6f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.448446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6987b56-5ed4-4455-952e-5cff68286a6f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.450123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 354/711] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:03.461695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.462026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.462259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.480759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.500910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:03.502149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:03.529269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 358/712] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:03.566926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.606699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 355/713] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:03.638349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.747032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c9053-33c0-4ee6-8d72-e943d8d48461 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.763003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1f2c6-6c9e-463d-86d9-c6dd7861eb11 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.784928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.799484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.800458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.801092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.801530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] PATH_INFO: `/v3/services/network` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.802720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.804788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.811262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:03.812303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:03.812723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:03.813144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:03.819722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:03.820817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:03.821877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 359/714] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:03.829249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.837048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:03.839236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:03.860100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:03.861119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:03.862059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:03.862059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:03.862059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:03.862059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:03.862059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 356/715] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:03.865197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:03.870859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:03.962355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72820bb-3966-4f19-9b83-d59cf5a142bd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:03.994281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d0f9-d699-457b-8956-4a717bdb0606 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:04.007294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:04.008230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:04.008648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:04.009085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:04.010337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] RBAC: Adding query filter params (name=network) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:04.010780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:04.012334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:04.017369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 360/716] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:04.023051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:04.033386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:04.034062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:04.034344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:04.034622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:04.035520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:04.035775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:04.037412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:04.050373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 357/717] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:04.056818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:04.098158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:04.140540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:04.165950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610784-b2b8-4988-9a8d-5a912d1a7407 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:04.204628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d57b3-fb12-49f7-8d25-e884666b732a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:04.246058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:04.247781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:04.249017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:04.249884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:04.251280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] RBAC: Adding query filter params (type=network) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:04.253862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:04.254783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:04.255279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:04.255748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:04.256305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:04.256917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:04.257747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56757c3-16f3-4acc-9590-ec7a4268855d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:04.258774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a7976-13f7-44e9-809a-d0cc99f758a5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:04.279917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 361/718] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:04.301405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 358/719] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:44:04 2022] POST /identity/v3/users => generated 254 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:05.516639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6b074-fdad-4856-9656-a94440d4ebb6 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:05.517210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6b074-fdad-4856-9656-a94440d4ebb6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:05.517210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6b074-fdad-4856-9656-a94440d4ebb6 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:05.518824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 362/720] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:05.524129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec3e-d66e-4b20-8177-55bd0ecf47e3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:05.524514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec3e-d66e-4b20-8177-55bd0ecf47e3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:05.524692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec3e-d66e-4b20-8177-55bd0ecf47e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:05.549345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-54c6ec3e-d66e-4b20-8177-55bd0ecf47e3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:05.550538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-54c6ec3e-d66e-4b20-8177-55bd0ecf47e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:05.617809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 359/721] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:44:05.641883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6b074-fdad-4856-9656-a94440d4ebb6 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:05.759200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce2d00-0f47-41ef-9632-80aec525b2a8 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:05.759924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce2d00-0f47-41ef-9632-80aec525b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:05.760259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce2d00-0f47-41ef-9632-80aec525b2a8 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:05.763121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 360/722] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:05.767354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:05.767583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:05.767861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:05.776865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6b074-fdad-4856-9656-a94440d4ebb6 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:05.799419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:05.801762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:05.816222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:05.846287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:05.846961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:05.847237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:05.847506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:05.848272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:05.849648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:05.873902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 363/723] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:05 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:05.962494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 361/724] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:05.988455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.006441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:06.101988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7310-0d31-476f-bacf-dd4ebe8c6e81 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.139734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.140639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.140929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.141208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.152261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:06.159928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:06.160881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:06.161738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:06.166165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 364/725] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:06.167666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.274799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb95ae-2fa2-4b6d-99e9-c570d9a2a515 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.310618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.311294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.311571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.311836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.312621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:06.312915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:06.314365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d77e42-86a3-4690-8dd9-50325b50f991 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:06.319825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:06.323955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 362/726] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:06.329524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.420705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:06.430079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895b82-b710-4d66-a7ec-766016de8541 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.430435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895b82-b710-4d66-a7ec-766016de8541 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.430647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c895b82-b710-4d66-a7ec-766016de8541 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.432695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 363/727] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:06.446222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.446511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.446613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8bce4-e203-4a96-898a-baa09c35b266 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.446793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.481799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.482609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.482893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.483169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.488948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:06.490164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:06.494258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:06.496032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:06.522262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:06.523668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:06.524593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:06.524593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:06.524593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:06.524593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:06.524593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 365/728] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:06.532862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.570046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 364/729] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:06.605886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.692325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2e963-f72e-4adf-a3b7-a5d15512402b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.725172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:06.735828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.736532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.736815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.737090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.737827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:06.738132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:06.739535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:06.758733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 366/730] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:06.769011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.798346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b4d8f-4032-4653-9402-c2c6488b68f5 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.829825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:06.836764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.838361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.838830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.839116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.850673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d6091ad5-0851-49e6-87d4-21b0e4dff6f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:06.858503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:06.859438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:06.860314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 365/731] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:06.862840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77b90-a884-4c15-b3d0-c34d49a1a54a None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.863400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77b90-a884-4c15-b3d0-c34d49a1a54a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.863969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77b90-a884-4c15-b3d0-c34d49a1a54a None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.869877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 366/732] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:06.870768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77b90-a884-4c15-b3d0-c34d49a1a54a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:06.901719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef146-5a2c-414d-be24-1920a71ea363 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.950740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:06.951483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.951762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.952555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.959813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:06.961525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-9cddca97-2f31-43c7-a378-b0cdef31d2ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:06.977359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:06.978367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:06.979544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 367/733] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:06.980685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:06.980685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:06.980955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:06.993582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb77b90-a884-4c15-b3d0-c34d49a1a54a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:06.999942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:07.001146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:07.035099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.036363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.036902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.037398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.038757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:07.039359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.041064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.046974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.053069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 367/734] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:07.055109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.078647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 368/735] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:44:07.079687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.111809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.173460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b006309-8cda-499d-b3cc-1842b945a4f9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.187242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac79956-132a-4972-a4d8-7ae093418838 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.208562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.209157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.209836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.210112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.210945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:07.211253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.212703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa99506-5204-470f-ab9b-d1392b587454 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.223664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.224516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.224793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.225063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 368/736] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:07.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1d86-0127-401f-a31f-586ed5c8fcfd None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1d86-0127-401f-a31f-586ed5c8fcfd None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1d86-0127-401f-a31f-586ed5c8fcfd None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 369/737] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:07.236344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.236422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e1d86-0127-401f-a31f-586ed5c8fcfd None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.236492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:07.256923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:07.257896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:07.258797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:07.258797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:07.258797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:07.258797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:07.258797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 369/738] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:07.260358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.289499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.357269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e1d86-0127-401f-a31f-586ed5c8fcfd None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.364182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f399b-0150-458e-a8fd-55500bf94adb None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.368681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.396316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.397085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.397369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.397642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.398356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.399819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fffcad0-c6bc-4d53-9174-53dd1c472035 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.402130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.403023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.403314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.403592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.415713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.417315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.420612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 370/739] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:07 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:07.421773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.422058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.422332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.441736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:07.443234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:07.444894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:07.445810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:07.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:07.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:07.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:07.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:07.449422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 370/740] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:07.450639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.550977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 371/741] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:07.552216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.563079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.595893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b1427-7872-4f26-9dfc-20d5aba3201c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.646000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.646704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.646984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.647275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.648351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:07.648351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.649729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.664668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 371/742] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:07.665890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.672229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.678864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9372a7-f8d7-46cf-ab5a-7911bf5ed2db None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.725577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.726255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.726790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.727073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.727854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:07.728196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.730483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.746980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 372/743] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:07.748063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.790398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc95f-6d73-4333-b836-825684cde528 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.841274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.842479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.843483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.844004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.855894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:07.862198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:07.873120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafad615-3ec0-4884-b762-f12db538813d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:07.873986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:07.874900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:07.875760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:07.882873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 372/744] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:07.882873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.923670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:07.924522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:07.924797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:07.925063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:07.933184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:07.934712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:07.952594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:07.953525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 373/745] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:07.955246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:07.975456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.014262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cee326-c767-4405-ae4f-3197818fd07f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.057206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.058049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.058852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.059134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.072448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a35b8-45a8-46d9-a8ea-6906db130e69 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.074085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.076766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:08.095989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:08.097009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:08.097950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:08.101269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 373/746] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:08.102817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.113905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.114584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.114863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.115135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.115928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:08.116208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.117587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.133408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 374/747] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:44:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:08.133408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.178629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.222791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.235117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c5e0-e923-4f95-ab91-13c35a58ebb2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.245533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9571904-c532-440d-97dd-414d8c167219 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.277218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.277701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.277985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.278272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.279133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:08.279401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.280969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.286811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.287785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.288299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.288780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.289886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:08.290457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.292353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.299801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 374/748] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:08.300699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.307265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 375/749] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:08.308403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.415475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d9f4-1d35-41f0-820f-cb791e3a5254 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.431365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cabc7-501d-4ddf-81e4-fdfa6e42277a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.433970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.452375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.453169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.453444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.453713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.463933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.465385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.469203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.469857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.470137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.470413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.479059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:08.483159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:08.484552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 375/750] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:08.485513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.547943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.549425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb008e28-761c-4e39-b3b3-73604c49d6aa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.586126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95218d5e-861a-46f9-883e-d4acabf28ae7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.623470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.624399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.624678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.624946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.627024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=d08cdb2d283143aa83b38bb3b04aee8c, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.628504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d00649-e062-455e-946f-18a7dd980e6d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.656171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 376/751] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:08 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:08.657784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda147-e0e2-46e5-9360-25c6ec73096d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.658119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda147-e0e2-46e5-9360-25c6ec73096d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.658338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbda147-e0e2-46e5-9360-25c6ec73096d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.672033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 377/752] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:08.672033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbda147-e0e2-46e5-9360-25c6ec73096d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.688085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 376/753] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:08.689599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0106-06d9-482e-8966-8510d387584c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.689888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0106-06d9-482e-8966-8510d387584c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.690162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0106-06d9-482e-8966-8510d387584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.707853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.723205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-42ed0106-06d9-482e-8966-8510d387584c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:08.724763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-42ed0106-06d9-482e-8966-8510d387584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:08.775479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.799229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbda147-e0e2-46e5-9360-25c6ec73096d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.830492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 377/754] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:08.831084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed0106-06d9-482e-8966-8510d387584c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.845767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:08.846441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:08.846968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:08.847248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:08.848007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:08.848778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:08.850456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:08.867679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 378/755] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:08.868998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:08.962424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed0106-06d9-482e-8966-8510d387584c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:08.974066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:08.986838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bebf8-dae5-46b0-9da6-bfbaa326d523 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.000706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.002079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.002539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.003099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.014668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.016282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.027317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.028247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.028531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.028807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.029471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.030864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:09.034889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:09.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:09.036701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:09.036701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:09.036701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:09.036701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:09.036701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 378/756] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:09.036975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:09.044252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:09.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:09.048272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:09.055895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 379/757] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Jul 14 10:44:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:09.055895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.069055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.155791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82588d1-eb6f-4f76-8e3b-68c232ae4309 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.165863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce599b57-638a-4d3b-88e5-d7b08be56c10 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.193630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.194369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.194650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.194925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.201877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.202896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.203261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.203552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.203737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.204484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.204863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.205074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.206423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.220439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 380/758] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.221368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:09.224455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:09.226534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:09.228769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:09.231413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 379/759] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:09.232348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.246353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.314705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.334563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10f01-49bd-4926-bf9d-8ce70483ad7c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.342254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd9b9-a1d2-4a04-a816-569f8870a5b0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.369985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.370763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.371042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.371315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.378264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.380040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.382418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.383342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.383826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.384321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.385767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.386331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.387930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.393277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 381/760] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.394272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:09.398598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:09.399495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:09.400507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:09.403977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 380/761] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:09.404825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.483540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.507572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed852f-3814-43d0-9f8f-78110db1cdc1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.510904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f811-cbf6-407c-9830-1e6e9da2bbb4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.542552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.543221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.543503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.543778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.544511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.544813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.546207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.549485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.550321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.550744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.551160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.552187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.552661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.554233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.557761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 381/762] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.558638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.562206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.576670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 382/763] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.577822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.666563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37aad9e-72b1-436d-aae5-4d9e75100ccc None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.687452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0cae3-75ad-4385-a885-65b0c16c023f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.703197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.703966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.704328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.704606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.705508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.705804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.707344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.725194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 382/764] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.725194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.728098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.728765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.729252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.729562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.730361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] RBAC: Adding query filter params (interface=public, service_id=2f7adc52164f4ec09627b2822ce2166c, region_id=RegionOne) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.730670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.732101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a10aca-7eb2-4709-9312-59b6f0b65f2b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.736847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.744890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 383/765] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/endpoints?service_id=2f7adc52164f4ec09627b2822ce2166c&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.746339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9847bf-a83c-4656-b209-6d82b889447d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.746627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9847bf-a83c-4656-b209-6d82b889447d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.746903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9847bf-a83c-4656-b209-6d82b889447d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.748176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 384/766] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:09.752266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9847bf-a83c-4656-b209-6d82b889447d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.798204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.840515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b996684-0279-48b6-96d5-3a66b4102436 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.852319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9847bf-a83c-4656-b209-6d82b889447d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:09.883761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.884442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.884720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.885409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.891713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:09.892406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:09.892699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:09.892963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:09.893791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=6f958e0eb8444871851792cb254b56a7) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:09.894530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:09.896014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:09.903087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 385/767] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/role_assignments?user.id=6f958e0eb8444871851792cb254b56a7&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:09.903991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:09.986235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:09.999238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.001120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.031030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc34324-e474-4ad6-b92e-02be2f550284 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.071785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.072501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.072806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.073353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.074048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.075440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ea5e5-13fb-4acb-a071-51d990e19c2d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.091931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 386/768] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:10.091931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.091931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.091931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.096027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:10.125887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:10.127113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:10.182891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 383/769] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:10.188992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.279064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 387/770] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:10.308725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.353245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb1cc1-e535-4ab7-a17b-6e04879377fd None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.368592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:10.394074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.395325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.395799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.396295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.407958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.410001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.415338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1becf-4c2d-487c-b639-8058fc431753 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:10.429758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:10.430723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:10.431633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:10.431633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:10.431633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:10.431633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:10.438565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 384/771] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:10.440252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.460299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:10.468029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.470166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.470452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.470723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.471384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.475904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:10.496419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:10.497477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:10.498334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:10.502336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 388/772] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:10.518591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.571028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01579a9f-ed2e-4a66-8549-114ca9babe42 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.618825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.619502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.619809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.620109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.620831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:10.621132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.622513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.655831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 385/773] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:10.657367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.717861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97796042-2875-46ff-813a-4823027c57a7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.785421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:10.785993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:10.802718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98cee-a673-4fb2-b4f8-3cb34d8a3071 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:10.810686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.811599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.812105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.812552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.819896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] RBAC: Adding query filter params (name=network) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:10.820513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.822167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008fd95-26fa-4eb0-80fd-ca4fba5d7bab None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.834738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 389/774] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:10.836141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2bdd-9f60-465c-94c3-eac26b079913 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.836428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2bdd-9f60-465c-94c3-eac26b079913 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.836704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac2bdd-9f60-465c-94c3-eac26b079913 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.837256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:10.838096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.838200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 390/775] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:10.838346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.838763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.842998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac2bdd-9f60-465c-94c3-eac26b079913 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.847268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:10.848778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-469784b9-3d4d-4f27-b182-316334ef6a59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:10.863237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:10.864188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:10.865489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 386/776] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:10.866155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:10.866377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:10.866787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:10.883612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:10.885175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:10.960968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 387/777] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:10.962141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:10.973574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac2bdd-9f60-465c-94c3-eac26b079913 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.016459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.017128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.017408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.017681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.018385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] RBAC: Adding query filter params (type=network) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.018688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.020510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.027291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 391/778] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.028404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.049775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.086947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a2805-5696-4c14-aa2f-cc5c11e6a350 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.093952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.128606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.129590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.130053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.130613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.131589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.132104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.133741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.152046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 388/779] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.152898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.169606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9744-facf-413f-b61e-17b46d73606e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.204177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.205028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.205313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.205585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.216616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:11.224397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:11.225347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:11.226193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 392/780] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:11.226904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.278461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b47dc-0820-42d5-9953-6a66c08feb6d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.311681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.323039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.323858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.324170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.324760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.325820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] RBAC: Adding query filter params (interface=public, service_id=555cfef70ebb4d48bb291c1821bfa785, region_id=RegionOne) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.326130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.328015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.333076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e663e6-1875-4d22-8471-71b6238bb179 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.337910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 389/781] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/endpoints?service_id=555cfef70ebb4d48bb291c1821bfa785&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.338700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.344699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.372966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.373916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.374382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.374846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.375949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=8728607736914f729f751bb1d47bbca9) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.376473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.378192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.384413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 393/782] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/role_assignments?user.id=8728607736914f729f751bb1d47bbca9&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.385904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.484679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31df5-a3ae-4ced-9630-fee2910c1f2f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.525776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516ead6-18f5-4b7b-b49f-fc5a265e287b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.530531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.531240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.531517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.531787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.532581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.532884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.534290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.546368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 390/783] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.554714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.593180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.594484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.594967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.595438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.596380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.598044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.611136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 394/784] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.614608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.668905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c34c-8b0e-4462-9668-6ff1aaa13490 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.705834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.706655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.706932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.707205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.719755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.721412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:11.740166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:11.741153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:11.742049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:11.742049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:11.742049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:11.742049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:11.742049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 391/785] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:11.747976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:11.824748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:11.889336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9d490-56fa-4298-a9c5-d6fba705533c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:11.937344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:11.938035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:11.938367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:11.938645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:11.939386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:11.939697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:11.941494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:11.954946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 395/786] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:11.965132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:12.090623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69071d30-d4f6-49bc-8a76-caf0fb8669bd None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:12.092974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:12.125465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:12.126270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.126553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.126826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] PATH_INFO: `/v3/users/nova` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.134631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:12.136554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-cae0da77-d56d-4939-98ae-4a7ef22fb01b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:12.150833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:12.151823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:12.153800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 392/787] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:12.158054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:12.329552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f9436-7058-4abe-8ae8-800273b78793 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:12.331172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:12.358362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1c41-1276-47f0-9f62-5f492f289c3c None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.359028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1c41-1276-47f0-9f62-5f492f289c3c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.359527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1c41-1276-47f0-9f62-5f492f289c3c None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.361401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 393/788] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:12.366550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.366831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.367104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.372586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:12.373269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.373548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.373819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.374669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:12.374980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:12.376470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:12.405247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:12.406382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:12.413679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 396/789] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:12.418714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:12.506416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 394/790] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:12.540831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:12.572136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b339b0-55c8-493f-9d33-824390716d0f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:12.613231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:12.613890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.614171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.614447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.642120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:12.695701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:12.695922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:12.698124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780c46f-6274-4feb-abd2-08b64696d40c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:12.709221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb447c-7616-4e15-9c9b-993220594be2 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:12.755133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:12.756473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.756850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.757213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.772376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-eff225f2-439f-49fd-8db1-2b0a675b562f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:12.782152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:12.783117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:12.783992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:12.787962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 395/791] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:12.789318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78da05-cca3-4e38-adc0-9d096ef67c29 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.789605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78da05-cca3-4e38-adc0-9d096ef67c29 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.789884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78da05-cca3-4e38-adc0-9d096ef67c29 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.791311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 396/792] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:12.792751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78da05-cca3-4e38-adc0-9d096ef67c29 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:12.824361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 397/793] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:44:12.826605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:12.827182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:12.827656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:12.846641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:12.848374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:12.908311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:12.929926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 398/794] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:12.931176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.036161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78da05-cca3-4e38-adc0-9d096ef67c29 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.071406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.072108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.072391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.072669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.073729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.074039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.075787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.082317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c4ad4-d93d-467a-aee6-116d3b36d9f0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.090702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 397/795] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.090702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.103944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.127108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.128348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.128851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.129341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.140625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.142461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:13.160938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:13.161851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:13.162751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:13.162751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:13.162751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:13.162751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:13.165164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 399/796] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:13.166404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.194742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117c51-3f9e-4997-b4e7-e9aecea09b01 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.208840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.229279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.230153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.230628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.230949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.231612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.233034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:13.247383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:13.248381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:44:13.249236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:13.252483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 398/797] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:13.252972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.279838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c7c5a-2fa2-4220-b3d7-41f226f99e4d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.317623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.318859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.319406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.319926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.331704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.333559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.338221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:13.353568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:13.355016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:13.355966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:13.355966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:13.355966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:13.355966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:13.358920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06174e6-e2b2-4bc6-9733-45ea247defdc None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.364386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 400/798] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:13.365972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.395787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.396487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.396761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.397037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.397758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.398053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.399453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.413277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 399/799] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.414166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.479184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.490148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd388d8a-1dda-4e7c-814d-b6ab2313cd38 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.517574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc12a2-b7be-4b77-88c1-1aa7fb94bfef None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.529026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.530009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.530486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.531141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.532175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.532675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.534245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.539961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 401/800] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.541131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.557451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.558134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.558485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.559009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.559465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.559774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.561424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.570504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.579046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 400/801] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.581113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.666617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6640f-d2f2-48ff-ab00-44e3fbe98af3 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.698786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c62542-2398-4bc5-9fab-16e9f86a8552 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.704003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.705121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.705715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.706202 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.714046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.716003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:13.731229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:13.732174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:13.733476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.734078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.734182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.734371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.735079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.735382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.736751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 402/802] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:13.736867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.738094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.746931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 401/803] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.747847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.756910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.813216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:13.852102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16169f3-7923-4978-af4b-7b917b6e474e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.854939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef0963-e0b8-4ea7-b912-3912cf017d91 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:13.887642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.888474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.888751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.889026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.890392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:13.891081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.891362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.891603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.892435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:13.892710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.894170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.897114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:13.898579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:13.911358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 402/804] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:13.911358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-29c30d11-92df-4985-8079-068d3b6ba5b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:13.915449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:13.916703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 403/805] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b27974-a319-4baa-a289-a1d30e77b516 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:13.918106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b27974-a319-4baa-a289-a1d30e77b516 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:13.918822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b27974-a319-4baa-a289-a1d30e77b516 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:13.922575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 404/806] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:13.923439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b27974-a319-4baa-a289-a1d30e77b516 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.004156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.019683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85237-7839-4c4c-90e2-8f3d6a50f936 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.030363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b27974-a319-4baa-a289-a1d30e77b516 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.043365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.054219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.054879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.055145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.055432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.056225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.057721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e12d0-0fd2-464a-ae2a-fdc6be0006f0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.064245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.064906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.065178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.065422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.066233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:14.066540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.068039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.075034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 405/807] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:13 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:14.079147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 403/808] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:14.080857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.081756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc449d-fab0-4c0f-b0a4-4c64be6a8a6f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.082161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc449d-fab0-4c0f-b0a4-4c64be6a8a6f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.082513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc449d-fab0-4c0f-b0a4-4c64be6a8a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.109599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9fbc449d-fab0-4c0f-b0a4-4c64be6a8a6f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:14.111157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9fbc449d-fab0-4c0f-b0a4-4c64be6a8a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:14.219719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 404/809] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:14.221501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28596a4-18f2-4b26-94c3-416fbe06bb8e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.222276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28596a4-18f2-4b26-94c3-416fbe06bb8e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.222637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28596a4-18f2-4b26-94c3-416fbe06bb8e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.228488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 405/810] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:14.230559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.230852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.231128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.238666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f0b6a-ba32-4a0e-aad3-74ff42a211fd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.264644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:14.264644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:14.286372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.287055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.287333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.287612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.288490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:14.288792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.290249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.300880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.314135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 406/811] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:14.315319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.347103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 406/812] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:14.348309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.395846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.434064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef17228-a038-4563-97b4-7a09cd33fa5e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.477716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.479150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.479836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.480327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.486273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae033b8-7d9d-4dc3-8719-82bc43ec9507 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.492523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:14.501218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:14.502168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:14.503021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:14.505818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 407/813] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:14.506846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.523932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.525100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.525622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.526133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.536955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.538723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:14.559194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:14.560195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:14.561086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:14.561086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:14.561086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:14.561086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:14.563538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 407/814] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:14.564680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.590685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.644113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e46cb-0a56-4903-86e9-7b50ab3ff565 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.672311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.686909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac91b2-3865-4cef-ba8f-faa36d8622fd None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.692810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.693891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.694883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.695332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] PATH_INFO: `/v3/services/network` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.696180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.698622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:14.714998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:14.716036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:44:14.716903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:14.718140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 408/815] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:14.718494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.730544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.732125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.732612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.733117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.734445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:14.735303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.737409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.749932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 408/816] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:14.751096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.837291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b213cc8-8f67-4d58-b14c-b8c43bc0f31b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.861878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:14.878289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.879009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.879293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.879567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.880315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:14.880618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.881781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8bfc-2a73-4bc2-9741-7a5a1bace2ec None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:14.881992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.895367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 409/817] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:14.896604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.947542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:14.948426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:14.948766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:14.949100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:14.949872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] RBAC: Adding query filter params (name=network) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:14.950237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:14.952890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:14.958750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 409/818] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:14.963924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:14.979576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.035100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1184e2d-e966-4be1-932d-7e682f967d8c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.068171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3d964-ccb7-4108-b7d7-4a0335e54893 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.078623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.079424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.079702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.080006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.093256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.095169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.104115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.105244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.105706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.106156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.113429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.115121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:15.115636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:15.116548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:15.117477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:15.117477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:15.117477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:15.117477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:15.117477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 410/819] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:15.118141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:15.133429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:15.134344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:15.135747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:15.143193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 410/820] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:15.144377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.152171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.232338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.236931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6286ca-2ccc-421a-b1e4-2a8ef855cce7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.268965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df0ef-857a-4470-ac62-7a8e9225934f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.279609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.280319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.280598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.280896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.282254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] RBAC: Adding query filter params (type=network) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:15.282562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.284576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.297340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 411/821] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:15.297340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.316977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.318107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.318588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.319058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.320047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:15.320572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.322185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.336132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 411/822] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:15.338151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.411802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.418757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a795a-200f-42ce-8409-f1fdb2b6d347 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.454525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625acd5-b705-4e2a-81dc-8e8718d0d37d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.455314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.456203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.456654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.457093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.458093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:15.458570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.460378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.477687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 412/823] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:15.478780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.490614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.491485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.491745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.492133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.493005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.494510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.498213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.523931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 412/824] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:15.526353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.688141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c663cbb-971b-44c3-b2cd-baa95081ea73 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.692727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e8878-49a5-4e28-9d65-6b69e8175f65 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.748354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.748731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.749288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.749566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.749691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.749853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/6f958e0eb8444871851792cb254b56a7/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.749945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.750113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.751868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=6f958e0eb8444871851792cb254b56a7, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.753626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.760932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.762605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.774120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 413/825] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:15 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/6f958e0eb8444871851792cb254b56a7/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:15.782642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:15.783682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:15.784689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 413/826] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:15.789980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:15.849174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:15.924011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ee014-92b7-4b43-a726-f861285277eb None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:15.966737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:15.967473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:15.967762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:15.968077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:15.968942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:15.969254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:15.970706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:15.990419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 414/827] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:15.996092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.016303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.100162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc998-1025-4bc4-87af-a0184e31afe2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.139925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.141105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.141584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.142053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.153858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.155627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-5d508892-a1a1-498b-9209-96a6ca4bf995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:16.173601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:16.174543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:16.175442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:16.175442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:16.175442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:16.175442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:16.177718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 414/828] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:16.185027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.256750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a124-1568-4c15-b666-230b2db3694b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.257366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a124-1568-4c15-b666-230b2db3694b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.257839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a124-1568-4c15-b666-230b2db3694b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.259709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 415/829] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:16.265495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df23c9-8c43-4819-be12-b8b4ba92a5d7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.266023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df23c9-8c43-4819-be12-b8b4ba92a5d7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.266498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df23c9-8c43-4819-be12-b8b4ba92a5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.283816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-66df23c9-8c43-4819-be12-b8b4ba92a5d7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:16.284179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5881-9f3e-4a85-aeaf-562ce038c5c5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.285558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-66df23c9-8c43-4819-be12-b8b4ba92a5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:16.292277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.320218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.320880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.321160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.321432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.322152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:16.322453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.324141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.340176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 415/830] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:16.341869 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.355974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.370817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 416/831] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:16.382703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c00f-1e2e-45fb-9b4e-e6a09bb4f2ac None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.383210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c00f-1e2e-45fb-9b4e-e6a09bb4f2ac None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.383488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c00f-1e2e-45fb-9b4e-e6a09bb4f2ac None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.385504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 417/832] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:16.390786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.391072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.391349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.409968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:16.411169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:16.448704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3520-6224-4ad9-905e-6c9aa9043b9a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.479937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 418/833] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:16.481301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.486763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.487819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.488350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.488822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.498434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.500753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:16.517135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:16.518035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:16.518932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 416/834] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:16.519426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.543213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.602025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d57f0-a0ff-42bd-8ea2-c6e7c48e984f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.619913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacecf7-60cc-4a4e-9c97-fcb843c3406f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.629416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.639375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc198-6731-4969-8fde-fdb789009def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.640216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc198-6731-4969-8fde-fdb789009def None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.640497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc198-6731-4969-8fde-fdb789009def None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.640769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc198-6731-4969-8fde-fdb789009def None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.641424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc198-6731-4969-8fde-fdb789009def None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.642830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc198-6731-4969-8fde-fdb789009def None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-524fc198-6731-4969-8fde-fdb789009def None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:16.655480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:16.656499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:16.657353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 419/835] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:16.658196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc198-6731-4969-8fde-fdb789009def None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.659152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.660007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.662384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.662676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.663341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.664829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:16.677520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:16.678648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:16.679516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 417/836] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:16.680388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.771463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.771670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc198-6731-4969-8fde-fdb789009def None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.784199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ba9e-8069-4276-84d5-b8ad03f2882a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.806189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.806842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.807162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.807408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.808384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:16.808693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.810189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.818994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.819662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.819970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.820247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.820953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:16.821255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.822757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 420/837] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:16.822864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.823430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.836608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 418/838] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:16.836608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:16.885599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:16.936977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d4ae5-6701-4622-91fb-e99200dab193 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.943027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a9494-55de-455f-bc66-51fa4c2b0863 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:16.978963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.979532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:16.980032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.980396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.980507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:16.980654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:16.980763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.980927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/8728607736914f729f751bb1d47bbca9/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:16.981559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:16.981925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:16.983378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:16.988977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 421/839] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:16.989666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:17.001706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=8728607736914f729f751bb1d47bbca9, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:17.003211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:17.019262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 419/840] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:16 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/8728607736914f729f751bb1d47bbca9/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:17.020303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:17.023053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:17.132334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48b8b-e0c6-4194-a4d5-8ee9576503af None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:17.138502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2406cbe-bc78-4cb4-945d-dbfb13276b70 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:17.157568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:17.183938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:17.184209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:17.184897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:17.185184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:17.185301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:17.185814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:17.186180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:17.186368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:17.186469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:17.186869 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:17.187726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:17.187989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30736852-d29c-4ec6-b8f6-2f9a72f9151e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:17.188334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:17.190243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba70b-9cc6-46e1-af75-c181fe193a71 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:17.194622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 422/841] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:17.197003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 420/842] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:17.911409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104098d-36d8-4ece-a50e-8cf2fd8e3b54 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:17.911765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104098d-36d8-4ece-a50e-8cf2fd8e3b54 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:17.912040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104098d-36d8-4ece-a50e-8cf2fd8e3b54 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:17.913682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 423/843] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:17.919294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:17.919579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:17.919858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:17.937809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:17.938691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:18.001464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:18.009754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 421/844] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:18.033445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104098d-36d8-4ece-a50e-8cf2fd8e3b54 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:18.136692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104098d-36d8-4ece-a50e-8cf2fd8e3b54 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:18.171715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:18.172473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:18.172753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:18.173028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:18.183676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:18.191572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:18.192524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:18.193357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:18.196134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 424/845] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:18.203149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:18.308639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ca7c-19ec-43da-bb26-8a7561a20c74 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:18.345590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:18.346330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:18.346618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:18.346893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:18.347665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:18.348006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:18.349445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:18.364080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 422/846] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:18.365694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:18.531988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:18.594465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9b05-fa97-4f77-aefd-07ab2bc58e87 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:18.629338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:18.630515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:18.630988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:18.631452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:18.643013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:18.644818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:18.663614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:18.664580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:18.665477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:18.665477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:18.665477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:18.665477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:18.667953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 425/847] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:18.673403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:18.734491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:18.778574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab484ac-d637-4024-985f-1c374666ffd5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:18.813416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:18.814007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:18.814291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:18.814572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:18.815307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:18.815616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:18.817033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:18.829905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 423/848] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:18.835523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:18.933804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:18.943263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b918d2-f5c3-41a6-b810-63e29fd5176b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:18.977812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:18.978673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:18.978955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:18.979232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:18.987487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:18.988976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:19.003368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:19.004366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:19.005704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 426/849] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:19.011087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.113761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc45b7-53b5-4a81-b578-4a4958d8928a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.136362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.149879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.150578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.150861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.151138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.152346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:19.152663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:19.154552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3103b-078d-409a-a706-00b3c76ac87f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.176740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 424/850] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:19.178719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.224381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16739e-5f24-421f-ae05-76da19ba6170 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.224904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16739e-5f24-421f-ae05-76da19ba6170 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.225157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16739e-5f24-421f-ae05-76da19ba6170 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.228217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 425/851] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:19.236891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692df6-f52f-4bb3-bb5a-65983e5654f7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.237400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692df6-f52f-4bb3-bb5a-65983e5654f7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.237874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5692df6-f52f-4bb3-bb5a-65983e5654f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.255969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b5692df6-f52f-4bb3-bb5a-65983e5654f7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:19.257250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b5692df6-f52f-4bb3-bb5a-65983e5654f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:19.288789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934fa08-cf46-4a2b-9bd3-c79a71346e5a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.323610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.324334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 426/852] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:19.324485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.324655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.324900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.325646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9e4c-857a-4437-bbfc-7e1459ff4a79 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.325929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9e4c-857a-4437-bbfc-7e1459ff4a79 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.326201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c9e4c-857a-4437-bbfc-7e1459ff4a79 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.327445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 427/853] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:19.328508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974fc6-2d70-4405-8091-5276b193bc55 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.328785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974fc6-2d70-4405-8091-5276b193bc55 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.329058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974fc6-2d70-4405-8091-5276b193bc55 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.331919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 428/854] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:19.333319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3509e-c26f-43c5-9656-78b5d2fa8d83 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.333591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3509e-c26f-43c5-9656-78b5d2fa8d83 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.333891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3509e-c26f-43c5-9656-78b5d2fa8d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.351499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-67f3509e-c26f-43c5-9656-78b5d2fa8d83 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:19.352743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-67f3509e-c26f-43c5-9656-78b5d2fa8d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:19.357478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.405046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:19.406421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.431898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 429/855] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:19.431898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.431898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.431898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.450632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:19.451840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:19.481517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.506985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 427/856] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:19.508082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.521194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 430/857] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:19.522039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.562945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.608010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734422c-a229-4818-90c4-99b294db23a7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.620789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c5009-a991-4a5f-881e-bb1bf68f5592 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.643618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.644305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.644579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.644844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.645579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:19.646964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.658705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.659444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.659727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.660038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.660573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 428/858] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:19 2022] POST /identity/v3/services => generated 269 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:19.661369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.671860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:19.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:19.680991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:19.682252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 431/859] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:19.683238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.715416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.794706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682f24e-08c6-4673-ba04-de1c2e82afc1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.805218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4028-497f-4dc6-91a5-3cedf206cb4c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:19.816163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:19.835950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.836745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.837008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.837528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.838000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:19.839550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.848768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:19.849761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:19.849938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:19.850177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:19.856336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 432/860] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:19 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:19.857209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:19.864945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:19.866657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:19.888935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:19.889909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:19.891122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:19.891122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:19.891122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:19.891122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:19.891122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 429/861] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:19.896225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.004482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.010710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115ed2-3097-4741-ba80-2c7483c9277c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.038021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e3bb2-596c-43b1-81ee-e94cb69b50da None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.062695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.063371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.064135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.064411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.065151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.065437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.066814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.079307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 430/862] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:20.083660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.084519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.084806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.085080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.085668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.085867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.086132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.087521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.097783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 433/863] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:44:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:20.103238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.110153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.192632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1432-574e-40a1-abd4-10f220b9f28f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.228548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.229802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.230340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.230801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.234713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02f88-895b-422b-9d9c-dac5afe53062 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.238795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.240597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:20.255031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:20.255987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:20.256893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:20.259484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 431/864] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:20.266071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.271308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.272172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.272451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.272721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.285917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.287804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.290552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:20.310272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:20.311475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:20.312410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:20.312410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:20.312410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:20.312410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:20.315945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 434/865] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:20.317731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.360293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.395144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1060e3-6625-45e5-945f-32c1233bdbad None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.440127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38d564-b8a0-48c3-8eb0-ed969f0901b4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.450793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.451409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.451688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.451988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.452777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.453368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.455110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.472011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 432/866] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:20.478680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.486852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.487522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.487806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.488123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.488855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.489158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.490540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.507422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 435/867] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:20.512634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.572837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.592274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba9a-aba0-4085-a98f-a8d3301fcf37 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.627980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933535-947e-45db-862d-16a36f516dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.628873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933535-947e-45db-862d-16a36f516dff None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.629258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933535-947e-45db-862d-16a36f516dff None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.629681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e933535-947e-45db-862d-16a36f516dff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.630577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933535-947e-45db-862d-16a36f516dff None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=6f958e0eb8444871851792cb254b56a7) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.630987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933535-947e-45db-862d-16a36f516dff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.632605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e933535-947e-45db-862d-16a36f516dff None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.636078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eefa3-df88-45d5-84c2-c95a5783352a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.645258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 433/868] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/role_assignments?user.id=6f958e0eb8444871851792cb254b56a7&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:20.679062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.680193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.680611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.681017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.689929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.691690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:20.714563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:20.716320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:20.718311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:20.727668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 436/869] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:20.729445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933535-947e-45db-862d-16a36f516dff None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:20.734959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:20.841317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e933535-947e-45db-862d-16a36f516dff None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:20.879584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:20.880540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:20.880958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:20.881394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:20.882369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:20.882813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:20.884408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:20.907604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 434/870] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:20.909249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.013217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac303-6d1b-43ea-ac10-00c42bb7ac79 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:21.032135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:21.052513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:21.053168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.053446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.053715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.124043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:21.125493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:21.230484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 437/871] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:20 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:21.231916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.295191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:21.333694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f1ad-230f-4b96-9e94-4fded1efba5c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:21.368082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:21.368883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.369206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.369480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.380207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:21.381688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:21.400198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:21.401174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:21.402232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:21.402232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:21.402232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:21.402232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:21.406157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 435/872] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:21.407582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.425889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:21.512580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a2f6b-3822-429b-b06b-1016ac9c9d7b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:21.547248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:21.547830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.548133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.548406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.549129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:21.549431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:21.550809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:21.567403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 438/873] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:21.568776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.670338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b64b55-619f-4b18-a87d-7de500291ef2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:21.684759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:21.705354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:21.706150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.706428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.706698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.713551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:21.715043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-1abd55a0-a197-42c5-ad7e-2c341a49f410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:21.728805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:21.730874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:21.733028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:21.739893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 436/874] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:21.740119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.780669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c543-f27d-42ac-a24a-3395496b24cf None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.781025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c543-f27d-42ac-a24a-3395496b24cf None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.781225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204c543-f27d-42ac-a24a-3395496b24cf None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.782854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 437/875] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:21.787769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.788095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.788377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.806167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:21.807387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:21.821547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:21.855440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b931-bda4-4326-8ca5-6665e1006ae7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:21.888153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 438/876] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:21.895817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:21.896512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:21.896790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:21.897062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:21.897848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:21.898154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:21.899585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:21.907389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:21.916287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 439/877] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:21.920939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.020752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad1189-5b39-41d3-9de3-3a689457ea09 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.023091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.028133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a72b6-d054-41c9-9fb7-abd90eb19481 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.060409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.061232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.061575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.061779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.062420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.063213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.063789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.063985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.064195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.064422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.065241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=8728607736914f729f751bb1d47bbca9) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.065541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.066960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.070603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-5ff6caac-bde6-45b6-b788-93235f607ed9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:22.076239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:22.077190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.083460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 439/878] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:22.084613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e72c-ec1b-4bd5-924b-130f9f75a18d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.084613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e72c-ec1b-4bd5-924b-130f9f75a18d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.084613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e72c-ec1b-4bd5-924b-130f9f75a18d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.084821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 440/879] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:21 2022] GET /identity/v3/role_assignments?user.id=8728607736914f729f751bb1d47bbca9&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:22.084821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.084951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 440/880] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:22.086905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.087190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.087460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.105672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:22.106894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:22.195535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 441/881] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:22.214779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd18eb-87e4-49d0-8d62-1955296278d4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.218638 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.264915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.265594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.265873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.266140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.266859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.267164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.268594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.275306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 441/882] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:22.281427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.344913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.351143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cd9b2-7616-40b7-955e-2312065a2a74 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.359630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.398209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.399479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.399815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.400133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.401073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.403011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.406919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97915-ddec-4e42-9aaf-978dd61b53e4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:22.418033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:22.419064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:22.419949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 442/883] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:22.427064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.448152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.449502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.449834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.450113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.450902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.451528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.452787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.459021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 442/884] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:22.465835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.530532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2633e1-2812-4249-9550-2d0ebca0b2e0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.565423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.566448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.567064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.567524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.568458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.568934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.570496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.579901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6e257-8fc7-4b3c-882d-d1829d671398 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.580363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 443/885] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:22.582020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.605827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.610299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.616696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.617451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.617746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.618032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.618864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] RBAC: Adding query filter params (interface=public, service_id=87916a45ad534614a8b1ad6da57bc044, region_id=RegionOne) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.619147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.620851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.632077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 443/886] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/endpoints?service_id=87916a45ad534614a8b1ad6da57bc044&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:22.633213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.689016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36683d-dc95-4d21-bffa-99328f1b4096 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.724280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.725306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.725796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.726258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.727185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.727587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.729144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.739314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 444/887] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:22.740602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.755981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ce1c-15c5-445d-9280-3afe09ca49a3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.789729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.790424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.790699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.790965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.791642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.793166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e23d9-411d-423c-93e2-1c4657490237 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.799069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 444/888] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:22.808138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a048f-839f-4028-bf16-8af650652d41 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.808138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a048f-839f-4028-bf16-8af650652d41 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.808138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a048f-839f-4028-bf16-8af650652d41 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.808138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 445/889] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:22.808138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.809259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.809259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.828278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:22.829981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:22.867946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.875036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97082135-859c-4aa6-8129-ef1f19f5dbd0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:22.904298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:22.915295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:22.916203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:22.919339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:22.919339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:22.919339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] RBAC: Adding query filter params (interface=public, service_id=acd7b5dbd93145f3890de533a138fe21, region_id=RegionOne) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:22.919339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:22.922115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:22.925212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 446/890] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:22.929507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 445/891] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/endpoints?service_id=acd7b5dbd93145f3890de533a138fe21&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:22.941071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:22.955236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:23.047228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94e52d-71b9-4922-a78d-21af2f89c068 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:23.080525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b5a12-d193-45a3-9bf0-f52f19360192 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:23.084542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:23.085214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:23.085493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:23.085762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:23.086443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:23.088372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:23.096787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 447/892] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:23.123764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:23.124562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:23.124838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:23.125109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:23.125778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:23.127218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:23.141500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:23.142587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:23.143479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 446/893] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:23.149106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:23.216288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:23.260007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868a956-f7ec-4fdf-ab83-45c781840745 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:23.296851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:23.297523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:23.297798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:23.298067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:23.298780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:23.299085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:23.300512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ff1a9-22a0-4335-9886-d6e60d93454e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:23.305827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 448/894] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:23.311598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:23.332979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:23.418098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246fbd-1ecb-43f7-9a76-b5a2b56c4d91 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:23.452127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:23.452796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:23.453070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:23.453340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:23.454044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:23.454345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:23.455717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4630381-4372-45f6-a05c-c2fc4444f9d2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:23.460825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 447/895] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:24.322880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b1a1-1a21-4e7b-9360-43d31d5aa0f9 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.323422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b1a1-1a21-4e7b-9360-43d31d5aa0f9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.323422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36b1a1-1a21-4e7b-9360-43d31d5aa0f9 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.325042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 449/896] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:24.330237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.330529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.330803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.347485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:24.348689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:24.416467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 448/897] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:24.439600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36b1a1-1a21-4e7b-9360-43d31d5aa0f9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:24.548568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36b1a1-1a21-4e7b-9360-43d31d5aa0f9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:24.563113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:24.584603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:24.585410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.585744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.586025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.598123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:24.606870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:24.607849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:24.608738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 450/898] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:24.614404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:24.632289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:24.714206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e2152-2637-4619-becb-f6f1f6373341 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:24.747748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:24.748439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.748713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.748980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.749804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:24.750108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:24.751484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c168d89-9a8f-458e-be6b-b8d748e1d7c5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:24.763965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 449/899] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:24.766103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:24.869518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7850c1f-e561-40c8-ba45-739ab1127a0f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:24.904085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:24.904888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.905168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.905442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.916295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:24.918130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:24.937343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:24.938308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:24.939217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:24.939217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:24.939217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:24.939217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:24.939217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 451/900] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:24.942187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacb6a-e699-498e-8e5b-acc069597b23 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.942477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacb6a-e699-498e-8e5b-acc069597b23 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.942754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacb6a-e699-498e-8e5b-acc069597b23 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.944563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:24.944689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 450/901] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:24.950063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:24.950348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:24.950627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:24.967812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:24.969039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:24.996319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.044144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 451/902] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:25.064798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.066735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c980b0-cb1d-49c7-83c2-12c52dde1018 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.068043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.109002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.110363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.110792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.111207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.112451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:25.114037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.115451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.129037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 452/903] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:25.137041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.189423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9407613-3122-43af-8d5d-3601ba436524 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.229984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.231332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.232261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.232957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.233383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.234198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.236236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.249482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214cde2-b846-4f24-bcda-47293fd9b017 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7c4e246c-c787-4bbb-854c-3b7e20228a53 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:25.254087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:25.255053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:25.255909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 452/904] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:25.257273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ab72-9451-475d-ad98-e4652c9ac5be None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.257273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ab72-9451-475d-ad98-e4652c9ac5be None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.259208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ab72-9451-475d-ad98-e4652c9ac5be None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.260095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 453/905] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:25.264376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1ab72-9451-475d-ad98-e4652c9ac5be None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.291490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.292787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.293065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.293331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.304676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.306138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-432a5caf-0d08-4124-b2a8-1193712ed6d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:25.321715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:25.322676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:25.324207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 453/906] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:25.324797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.324864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.325435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.336626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.343376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:25.344608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:25.372531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1ab72-9451-475d-ad98-e4652c9ac5be None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.409252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.410157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.410599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.410877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.411622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:25.411948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.413324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.418665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 454/907] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:25.419160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 454/908] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:25.419413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.433286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.514882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.528601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd650fe-965a-49bd-85c9-97e41c6ac931 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.541935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba106b5a-c121-4856-940e-4c895a85a046 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.558045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.569048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.569725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.570004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.570276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.571121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:25.571429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.572903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd885108-1bf1-4311-9577-61770a86806e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.578666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f205208-5636-4713-9a66-6899b296843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.579333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f205208-5636-4713-9a66-6899b296843f None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.579611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f205208-5636-4713-9a66-6899b296843f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.579905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f205208-5636-4713-9a66-6899b296843f None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.580607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f205208-5636-4713-9a66-6899b296843f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:25.581011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f205208-5636-4713-9a66-6899b296843f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.582391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f205208-5636-4713-9a66-6899b296843f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.587282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 455/909] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:25.588094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f205208-5636-4713-9a66-6899b296843f None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.591707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 455/910] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:25.593215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5da3-a9d9-4d56-ae53-dd3f8d86ee77 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.593736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5da3-a9d9-4d56-ae53-dd3f8d86ee77 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.594126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef5da3-a9d9-4d56-ae53-dd3f8d86ee77 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.596979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 456/911] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:25.597793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef5da3-a9d9-4d56-ae53-dd3f8d86ee77 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.708798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f205208-5636-4713-9a66-6899b296843f None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.712001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef5da3-a9d9-4d56-ae53-dd3f8d86ee77 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.743788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.744616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.744896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.745166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.745852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.746816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.747226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.747599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.747986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.748262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.749168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:25.750811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abc8f2-9826-4208-bd37-5efbf2ea3425 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:25.758722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:25.759712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:44:25.760596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:25.764245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 456/912] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:25.765089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.769233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.787902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 457/913] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:25.787902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.787902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.787902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:25.794233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:25.795466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:25.844913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:25.881546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 458/914] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:25.882496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:25.902037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ed45-3f37-4705-a1d6-132619a02621 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:25.940754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:25.941497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:25.941769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:25.941963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.011793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd75c89-727c-448c-9aae-0feee27c50db None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.016901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.018411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.022666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.048019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.048685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.048970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.049240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.049956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:26.050262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.051652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.061557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 459/915] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:26.061557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.100810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.134046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 457/916] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:25 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:26.135047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.174166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e293b8-5715-43b6-ae9d-e95884b17959 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.210295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.210953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.211237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.212138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.212246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.213623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.229360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 460/917] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:26 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:26.230282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.263163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523880e-ae78-47ac-aa4f-44e922ebf9f8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.267647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.327547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.328282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.328530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.328866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.329585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:26.329886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.331323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.342290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 458/918] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:26.343606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.360180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.368845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf835c-5426-4018-9641-50b28c7c6b47 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.411118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.411956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.412238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.412512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.426177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.427789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:26.445770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:26.446728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:26.447775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:26.447775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:26.447775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:26.447775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:26.447775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 461/919] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:26.452820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.481004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ba151-db88-4cf2-94d1-16dca0b9c9c4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.530139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.530758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.531279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.531560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.533217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.534714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.561969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 459/920] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:26.566871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc6564-eba7-48ca-a4ab-6e74c3733bf9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.614669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.615352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.615632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.615997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.616744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:26.617052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.618498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.633213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 462/921] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:26.639646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.664910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.749540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fac5b3-ecfa-4691-8362-b5bd21138308 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.785955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.786579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.786863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.787130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.794424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.795901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:26.810335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:26.811294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:26.812274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 460/922] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:26.817654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:26.822089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:26.925081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0898-5860-4f5c-ab92-eb526c20874c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:26.966289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:26.967308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:26.967792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:26.968404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:26.969423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:26.969950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:26.971609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08473e61-38d8-4ae3-8f67-077d39719c72 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:26.984817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 463/923] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:26.990624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:27.082430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:27.102780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbf59-25b5-4ea3-a4f2-34d3965ef9ad None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:27.139189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:27.140272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:27.140780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:27.141267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:27.142346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] RBAC: Adding query filter params (role.id=ad041f76f17d4a19aaf21217fc3fcc78, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=8728607736914f729f751bb1d47bbca9) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:27.142858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:27.144543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075d93-ac58-419e-8993-5f1c8b5082d1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:27.151172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 461/924] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:26 2022] GET /identity/v3/role_assignments?user.id=8728607736914f729f751bb1d47bbca9&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=ad041f76f17d4a19aaf21217fc3fcc78 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:27.909189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653d8c-62d3-427f-81eb-fa3e39ff991c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:27.909760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653d8c-62d3-427f-81eb-fa3e39ff991c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:27.909760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653d8c-62d3-427f-81eb-fa3e39ff991c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:27.911357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 464/925] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:27.918664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:27.919097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:27.919197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:27.936307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:27.937494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:28.006680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 462/926] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:28.028834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61653d8c-62d3-427f-81eb-fa3e39ff991c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.095480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:28.134974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61653d8c-62d3-427f-81eb-fa3e39ff991c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:28.170021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:28.170747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.171028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.171303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.171992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:28.173381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:28.185282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:28.186280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.187151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 465/927] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:28.192628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.277640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:28.303536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3adc0d-5f4d-4f69-b784-1c81449964f8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:28.338555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:28.339286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.339574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.339855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.340615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:28.340922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:28.342310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7ae91-89d4-4dc3-886f-264868c73cf5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:28.347718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 463/928] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:28.353366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.460631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d3e3a-5b68-4fd2-892d-5c4652cdd7a7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:28.460818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e998-936e-47bb-8bc1-e0d22f7448f8 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.461207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e998-936e-47bb-8bc1-e0d22f7448f8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.461491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e998-936e-47bb-8bc1-e0d22f7448f8 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.466614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 464/929] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:28.468241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.468524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.468801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.485543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:28.486740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:28.492371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:28.501112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:28.501902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.502174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.502458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.503814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] RBAC: Adding query filter params (type=compute) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:28.505300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:28.506695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:28.521233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 466/930] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:28.572289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 465/931] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:28.596934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.712229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78b49-c40b-42f2-a8a7-7bd21e09d38e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:28.746940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:28.747750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.748051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.748321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.748968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:28.750345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-11b2a3d6-4c7b-4d20-b8c9-71c4cab01034 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:28.763213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:28.764373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:28.765290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 467/932] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:28.770698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.840488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:28.864464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5f71c-267e-4085-8876-c5075c3b7eb3 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.865101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5f71c-267e-4085-8876-c5075c3b7eb3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.865574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5f71c-267e-4085-8876-c5075c3b7eb3 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.867348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 468/933] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:28.872798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.873408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.873908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.875708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba162a2-dc6a-4187-b3c4-3a2cfd3e1268 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:28.893129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:28.894737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:28.915541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:28.916613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:28.917102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:28.917664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:28.918807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:28.919359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:28.920808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:28.926504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 466/934] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:28.932547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:28.965745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:28.989420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 469/935] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:29.012811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.068346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327e4ac-7abf-464e-af89-b449699104bc None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.108446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.114412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.115634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.116051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.116763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.117682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:29.118289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.119772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.131071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 467/936] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:29.132318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.137648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56abe2-7649-42b1-b3f6-099f93043aa9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.178577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.179430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.179807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.180195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.180996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.182592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.184914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING keystone.server.flask.common [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:44:29.207795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.sql.core [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:44:29.207795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:44:29.207795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [parameters: {'id': '5642612a5ac74277a9351e8058ae72fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:44:29.207795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=95328) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-53ddb38f-d2d8-46fa-83ac-6832960e085a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5642612a5ac74277a9351e8058ae72fb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5642612a5ac74277a9351e8058ae72fb. Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:44:29.216510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result.read() Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 10:44:29.217327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application yield resource Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise self.value Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application session.commit() Jul 14 10:44:29.218360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:44:29.219288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:44:29.220263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application result.read() Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:44:29.221098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [parameters: {'id': '5642612a5ac74277a9351e8058ae72fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.221983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:29.225392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5642612a5ac74277a9351e8058ae72fb. Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 470/937] 10.222.0.27 () {60 vars in 1140 bytes} [Thu Jul 14 10:44:29 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb90ed-db7e-42e7-bf27-da5328e1b56c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.227846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb90ed-db7e-42e7-bf27-da5328e1b56c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.229404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb90ed-db7e-42e7-bf27-da5328e1b56c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.229404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 471/938] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:29.229404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb90ed-db7e-42e7-bf27-da5328e1b56c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.259932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471fccf-28a3-40de-b78c-4c24b39f6203 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.275336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.296660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.297320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.297593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.297864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.298653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] RBAC: Adding query filter params (interface=public, service_id=330412734fe644c884378fc8f81877bc, region_id=RegionOne) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:29.298960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.300368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a477af-e11f-496b-91a5-593fda79b181 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.305650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 468/939] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/endpoints?service_id=330412734fe644c884378fc8f81877bc&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:29.311131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.311131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.311131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.323948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:29.325149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:29.351191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb90ed-db7e-42e7-bf27-da5328e1b56c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.383372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.390609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.391416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.391695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.392184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.396738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 469/940] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:29.399035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.400664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.401128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:29.415494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:29.416555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.417609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 472/941] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:29.421981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.513203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7c4bf-a8ec-46f0-b888-06c88db7f80e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.526050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd6bb-a87e-4061-bf9b-7ee40cf4ad72 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.528309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.549764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.550432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.550712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.550986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.551665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.553093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.560896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 470/942] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:29.569204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.570083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.571463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.572074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.572608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.589451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:29.601847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:29.603224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:29.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:29.606923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 473/943] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:29.613289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.648394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.699419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dc38f-6813-4746-84f0-00e98b591636 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.737205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceb06e-9df9-474f-8e7a-07b01d39d319 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.738225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.738896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.739171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.739443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.740237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:29.740537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.741987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.760150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 471/944] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:29.782716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.783340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.783616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.783923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.784692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:29.784997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.786430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.797674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 474/945] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:29.803345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:29.914874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cca75-b055-433a-a6e5-ab11fad94fab None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:29.940904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:29.966212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:29.967032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:29.967314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:29.967590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:29.979060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:29.980618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:29.999045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:30.000053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:30.000952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:30.000952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:30.000952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:30.000952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:30.000952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 472/946] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:30.006144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.034842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:30.114943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad276-d488-45be-8f02-c8e12be8fc7b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.150662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.151337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.151646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.151947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.152673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:30.152984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.154385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.167590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 475/947] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:30.173267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.279865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15781d7b-e418-420a-ac6f-ce23b445f36a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.314708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.315438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.315761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.316072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.323488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.325086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:30.344598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:30.345623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:30.346602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 473/948] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:30.352116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.442339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:30.458159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03805b8a-54b0-4b6c-a396-3c382f1711ca None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.493114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.493783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.494064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.494335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.495171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:30.495535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.497002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d42d4-12ea-4ca1-a829-3323928f4f62 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.515377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 476/949] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:30.524966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.554211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:30.631282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d291d9f-4cc7-4be0-8e02-ab5d496b3774 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.647129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbab8-c071-4d2b-b469-40cd9fa40aa6 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.647451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbab8-c071-4d2b-b469-40cd9fa40aa6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.647704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcbab8-c071-4d2b-b469-40cd9fa40aa6 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.649496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 477/950] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:30.654811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.655096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.655375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.669457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.670264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.670546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.670811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.673098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:30.674630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:30.682393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.684130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:30.706264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:30.707203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:30.708131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:30.708131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:30.708131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:30.708131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:30.708131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 474/951] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:30.715922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.769033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 478/952] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:30.791218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.800804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:30.824757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5956d6-eb35-44f8-b799-6c4af4f04035 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.864055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.864731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.865018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.865291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.866022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:30.866325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.867774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.870872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:30.880415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 475/953] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:30.889804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:30.910054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5277ec0-434e-4b8e-975e-890951b623ba None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:30.947719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:30.948213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:30.948492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:30.948907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:30.949611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:30.951089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:30.972555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 479/954] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:30 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:44:31.025870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bde25d-0b3e-4b0c-b6a7-85ac5908813e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:31.078574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:31.079657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.080116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.080566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.083005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:31.089537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:31.091797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:31.108745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:31.109920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:31.111097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:31.113417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 476/955] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:31.121372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:31.227855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eab9a-e4d0-4999-9a7e-d953728f8a54 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:31.257910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:31.264236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:31.265333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.265800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.266082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.266863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:31.267171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:31.268644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628db0-986a-44e3-bd0d-9b224e6f01ff None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:31.281612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 480/956] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:31.287510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:31.392559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8929c2-30ff-447d-82dd-0545c513f756 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:31.428742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:31.429957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.430418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.430895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/8728607736914f729f751bb1d47bbca9/roles/ad041f76f17d4a19aaf21217fc3fcc78` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.433549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=8728607736914f729f751bb1d47bbca9, role_id=ad041f76f17d4a19aaf21217fc3fcc78)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:31.435203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396583-5e31-4d88-8dc9-18940760194d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:31.453253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 477/957] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:31 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/8728607736914f729f751bb1d47bbca9/roles/ad041f76f17d4a19aaf21217fc3fcc78 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:31.835324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215edf-90df-4368-a919-4ed13150f9cb None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.835648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215edf-90df-4368-a919-4ed13150f9cb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.836123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215edf-90df-4368-a919-4ed13150f9cb None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.837854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 481/958] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:31.843195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8091-4af8-41c3-b587-7d0e3b166f52 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.843566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8091-4af8-41c3-b587-7d0e3b166f52 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.843928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8091-4af8-41c3-b587-7d0e3b166f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.861651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b96b8091-4af8-41c3-b587-7d0e3b166f52 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:31.862830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b96b8091-4af8-41c3-b587-7d0e3b166f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:31.915233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:31.937851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 478/959] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:31.958902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215edf-90df-4368-a919-4ed13150f9cb None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:31.964740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b976b9-6c3e-4184-9afd-9379a7646a6e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.965067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b976b9-6c3e-4184-9afd-9379a7646a6e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.965289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b976b9-6c3e-4184-9afd-9379a7646a6e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.966841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 479/960] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:31.971637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:31.971966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:31.972247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:31.990401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:31.991588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:32.084167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 480/961] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:32.094522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215edf-90df-4368-a919-4ed13150f9cb None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.105785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.137629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.139736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.140513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.141080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.141939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.144707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:32.160218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:32.161321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:32.162341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 482/962] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:32.173508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.218204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28025a1f-1422-4a5a-80aa-80006c822a01 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.234282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:32.253519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:32.259437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.260564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.261006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.261840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.262825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.264549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.266050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING keystone.server.flask.common [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:44:32.280924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.sql.core [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:44:32.280924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:44:32.280924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [parameters: {'user_id': '05a995c3574b47e180e16c61f2c2f992', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:44:32.280924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=95327) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application result.read() Jul 14 10:44:32.300603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:44:32.301692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application yield resource Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise self.value Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application session.commit() Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:44:32.302844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:44:32.303855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:44:32.304852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application result.read() Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:44:32.305712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [parameters: {'user_id': '05a995c3574b47e180e16c61f2c2f992', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:32.306582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 10:44:32.307392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:44:32.308431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.308894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057bd08-bac3-4cb5-905e-cd6699d09529 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.315792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 481/963] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Jul 14 10:44:32 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:32.317392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.335902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.336579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.336782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.337035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.337738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:32.338092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.339579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.354841 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 483/964] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:32.361562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.450217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680834de-98e3-469d-8afd-bd45918dc38b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.464444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ca139-dfe5-423f-917c-aed4ffcb901e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.489886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:32.492871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.493685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.494146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.494440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.501108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.502089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.502560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.503002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.503392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.504189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:32.504718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.505342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.506428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.516205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 484/965] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:32.518807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:32.521255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:32.522165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:44:32.523117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:32.527371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 482/966] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:32.529795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:32.569804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:32.640836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1b6aa-7cce-432f-998d-5a30bdad893c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.642220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afcc9c-788a-4f6e-92ed-0f4ba8c8fc0a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:32.680687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.681681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:32.682591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.683505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:32.684022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.684255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:32.684536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.684926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:32.685362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:32.685668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.686310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:32.687108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd2b6c-2c4b-4bf2-8398-f2dd5a3bf586 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.688090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efc190-7d0a-40f6-9aa9-4121cc7e8614 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:32.714804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 485/967] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:32.745456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 483/968] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Jul 14 10:44:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:33.134573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc0ac-3e07-4fe4-ba29-dd14e261a4cf None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.135006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc0ac-3e07-4fe4-ba29-dd14e261a4cf None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.135264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc0ac-3e07-4fe4-ba29-dd14e261a4cf None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.137170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 486/969] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:33.142832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.143300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.143376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.160751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:33.162253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:33.241356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 484/970] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:33.264395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc0ac-3e07-4fe4-ba29-dd14e261a4cf None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.311397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.375490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dc0ac-3e07-4fe4-ba29-dd14e261a4cf None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.411622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.412451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.412734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.413007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] PATH_INFO: `/v3/services/compute` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.413720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:33.415166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-160459aa-50e1-43b9-999f-8da2b5aa9e92 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:33.427285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:33.428335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:33.434283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 487/971] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:33.436750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.460032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.489460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f06d-4119-4e32-8ec6-e16e74e6c47b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.489719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f06d-4119-4e32-8ec6-e16e74e6c47b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.490070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f06d-4119-4e32-8ec6-e16e74e6c47b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.491756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 488/972] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:33.496780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.497040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.497332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.517369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:33.518943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:33.542749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bf4e3-c21f-4cda-904c-1295b63f0e59 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.577797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.578470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.578754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.579026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.579735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] RBAC: Adding query filter params (name=compute) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:33.580077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:33.581459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:33.586231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 485/973] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:33.592121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.593738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 489/974] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:33.617347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.653250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.689041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.705850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dec0d-a463-4e62-9fdb-6cac735d353d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.724436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf5ecf-02f7-40a8-8e7e-f4f94c0f8bc8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.742009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.742730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.743174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.743550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.744407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] RBAC: Adding query filter params (type=compute) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:33.744816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:33.746639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:33.752198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 486/975] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:33.757639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.766501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.767524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.767810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.768124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.779217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:33.787259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:33.788416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:33.789471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:33.794047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 490/976] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:33.794722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.868276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55bd1d-6d8d-4708-8464-a4ac04635095 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.893785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.901354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8801-1d15-40de-8e5b-d8d16b645423 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:33.909690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.910388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.910671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.910947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.912151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] RBAC: Adding query filter params (interface=public, service_id=4b3fcc35199b4db3a753403eb5ca471b, region_id=RegionOne) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:33.912472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:33.914465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:33.920540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 487/977] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/endpoints?service_id=4b3fcc35199b4db3a753403eb5ca471b&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:33.925934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:33.942060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:33.948082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:33.948909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:33.949703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:33.949978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:33.950741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:33.951051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:33.952472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:33.961522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 491/978] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:33.966881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.044487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f296ef-efa1-42a9-9942-20be12b7127a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.068026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a20501a-aee9-4c35-9c6b-7026b3adf9ae None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.080170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.080839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.081225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.081499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.082297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.083671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.099910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 488/979] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:34.112944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.113866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.114221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.114566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.127269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.128896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:34.148267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:34.149282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:34.155287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:34.155287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:34.155287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:34.155287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:34.155287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 492/980] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:34.156951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.245245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:34.272673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d6d9d-90ee-4916-be88-f75518c00a19 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.306606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.307191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.307471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.307743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.308720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:34.309028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.310397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.322060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 489/981] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:34.328622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.343059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:34.437439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e34ec-1002-4e15-9932-5ce36139d16f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.471222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.472072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.472357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.472636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.481068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.482533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:34.497267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:34.498311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:44:34.499388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:34.503320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 493/982] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:44:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:34.504400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.610306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0796e0-4adf-4b82-be5f-c5f24e759e01 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.645946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.646626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.646918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.647193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.648069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:34.648383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.649814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dc056-e57e-4360-a73d-aa6f2bb27623 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.652571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:34.669284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 490/983] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Jul 14 10:44:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:34.674915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.745168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:34.781530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64dcec-7774-45a1-9c91-a6123d0b2fb8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:34.805766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedaea7-92ce-49de-9f94-be8ac2e3720f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.806066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedaea7-92ce-49de-9f94-be8ac2e3720f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.806341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedaea7-92ce-49de-9f94-be8ac2e3720f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.807852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 491/984] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:34.812941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.813237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.813498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.821620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:34.822505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:34.822790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:34.823062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:34.831467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:34.832784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:34.903188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:34.904704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:34.907995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 492/985] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:34.930807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:34.983358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.020550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 494/986] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:34 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:35.026005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.039947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4066de9-a5b5-43e6-b424-26f247fef404 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.065320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.077865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.078676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.078955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.079639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.080407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.081727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:35.094113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:35.095131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:35.096036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 493/987] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:35.101340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.131157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ca410-690f-4f1a-a89c-7c963777477a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.169315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.170032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.170309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.170660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.181545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.182511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:35.201629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:35.202631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:35.203620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:35.203620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:35.203620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:35.203620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:35.203620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 495/988] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:35.208542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.216992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d4c7-ed72-4c0e-b00c-7e2ea1c0ecef None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.231723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.253295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acf915-c967-4734-a865-567e556fceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.253971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acf915-c967-4734-a865-567e556fceaf None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.254243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acf915-c967-4734-a865-567e556fceaf None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.254533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79acf915-c967-4734-a865-567e556fceaf None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.255240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79acf915-c967-4734-a865-567e556fceaf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.255543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79acf915-c967-4734-a865-567e556fceaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.257033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79acf915-c967-4734-a865-567e556fceaf None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.262935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 494/989] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:35.268593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acf915-c967-4734-a865-567e556fceaf None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.314333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549a03d-4cfd-416d-8a35-9ea5fb3e3fc3 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.321473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.353133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.353817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.354124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.354391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.355111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.355418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.356811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.373088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 496/990] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:35.374099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.377069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79acf915-c967-4734-a865-567e556fceaf None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.419752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.420486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.420839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.421186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.421970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.422341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.423941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.429522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 495/991] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:35.435058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.462635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.482761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acc93e-e306-42f9-8f08-2a1a2122e1aa None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.516832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.517719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.518076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.518417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.525701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.527306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:35.542158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:35.543913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:35.545023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:35.548407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 497/992] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:35.549269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.554610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d2414-27a6-4bfc-a56a-f18b721b19be None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.572577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.593380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.594127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.594468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.594903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.595775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] RBAC: Adding query filter params (interface=admin, service_id=330412734fe644c884378fc8f81877bc, region_id=RegionOne) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.596172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.597720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.603341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 496/993] 10.222.0.27 () {56 vars in 1249 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/endpoints?service_id=330412734fe644c884378fc8f81877bc&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:35.609155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.655129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293e35-05a8-4073-aadc-3e24a5193cac None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.689397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.690066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.690402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.690731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.691564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.691962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.693484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.702325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:35.707452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 498/994] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:35.713252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:35.721457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a613df-b738-4f20-a10c-620961d1fc05 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.756324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.757075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.757420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.757755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.758503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.759985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2fb12-54f1-4530-9d03-85d95160f5f7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.775914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 497/995] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:35.833319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a6555-ab18-4bee-ad79-f6d1f9dd1768 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:35.871805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:35.872521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:35.872803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:35.873082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:35.873924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] RBAC: Adding query filter params (role.id=ad041f76f17d4a19aaf21217fc3fcc78, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=8728607736914f729f751bb1d47bbca9) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:35.874236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:35.875698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b586cf5-7200-4a7e-82c5-47044e988233 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:35.882684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 499/996] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:35 2022] GET /identity/v3/role_assignments?user.id=8728607736914f729f751bb1d47bbca9&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=ad041f76f17d4a19aaf21217fc3fcc78 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:36.115091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843923b1-2f4c-4ce5-8a3c-757671058b99 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.115375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843923b1-2f4c-4ce5-8a3c-757671058b99 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.115649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843923b1-2f4c-4ce5-8a3c-757671058b99 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.117213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 498/997] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:36.122265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.122548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.122830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.139808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:36.140999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:36.203798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:36.213813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 500/998] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:36.237410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843923b1-2f4c-4ce5-8a3c-757671058b99 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:36.288925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:36.349839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843923b1-2f4c-4ce5-8a3c-757671058b99 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:36.386924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:36.387836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.388157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.388431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] PATH_INFO: `/v3/services/compute` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.389096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:36.390479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:36.402728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:36.403730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:44:36.404912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:36.408207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 499/999] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:36.409917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:36.519926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581ed97-9784-4da5-8983-3f86a409b6bd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:36.555191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:36.555894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.556230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.556480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.557257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:36.557566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:36.558983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:36.568529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 501/1000] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:36.569772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:36.672910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efd5fb-6fbb-4f00-95bc-376146d4fb50 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:36.681974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:36.707733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:36.708428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.708714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.709045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.709760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] RBAC: Adding query filter params (type=compute) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:36.710064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:36.711444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc112d-df5f-44e7-8cbb-a5ddb52b6915 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:36.720997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 500/1001] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Jul 14 10:44:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:36.722277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:36.784045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:36.827307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d36b-452d-4c04-be9d-066b91b8bf9a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:36.865570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:36.866229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:36.866516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:36.866794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:36.867597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:36.869037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf46dd4-be84-464a-a8f3-70c349e74d52 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:36.892326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 502/1002] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:37.805112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617dec-c3da-474f-b225-4e7247ef4dd5 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:37.805654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617dec-c3da-474f-b225-4e7247ef4dd5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:37.805654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617dec-c3da-474f-b225-4e7247ef4dd5 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:37.807093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 501/1003] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:37.812670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab83b-32aa-4291-bc62-970819a24fff None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:37.813121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab83b-32aa-4291-bc62-970819a24fff None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:37.813210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaab83b-32aa-4291-bc62-970819a24fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:37.832667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cfaab83b-32aa-4291-bc62-970819a24fff None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:37.833997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cfaab83b-32aa-4291-bc62-970819a24fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:37.915211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 503/1004] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:37 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:37.938173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d617dec-c3da-474f-b225-4e7247ef4dd5 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:37.944843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7a6f-d07f-4e93-ab3d-75088e1c00e6 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:37.945134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7a6f-d07f-4e93-ab3d-75088e1c00e6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:37.945411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd7a6f-d07f-4e93-ab3d-75088e1c00e6 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:37.946830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 504/1005] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:37.951867 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:37.952220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:37.952602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:37.969899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:37.971097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:38.009001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:38.046460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 505/1006] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:37 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:38.051671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d617dec-c3da-474f-b225-4e7247ef4dd5 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.068689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.089066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.090275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.090557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.090833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.091626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.092903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:38.106021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:38.107006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:38.107999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 502/1007] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Jul 14 10:44:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:38.113357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.151713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:38.179262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50634956-63a3-467a-b8e7-b57df104a63f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.214050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.214894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.215177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.215451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] PATH_INFO: `/v3/services/image` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.216135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.217526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.219534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf0453-5c1a-4bc6-ab6b-acc6c598bfb8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:38.229050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:38.230057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:38.231140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 506/1008] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Jul 14 10:44:38 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:38.236691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.257398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.258069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.258351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.258618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.259735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:38.260371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.262332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.267789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 503/1009] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:38.273370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.277663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:38.346954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729f88-2e65-4964-95f7-5d36e5396c30 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.376451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:38.380192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c7ed0-872d-4d9a-9e69-82f5ee25c932 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.380444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.381203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.381479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.381751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.382457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:38.382760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.384213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.389421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 507/1010] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:38.395043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.421755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.422420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.422702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.422974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.423675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:38.424036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.425431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.430648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 504/1011] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Jul 14 10:44:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:38.436255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:38.495014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:38.499750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997864-7dd7-4f1d-b73b-c6212f4a57e2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.536087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.536748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.537054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.537291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.538017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] RBAC: Adding query filter params (type=image) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:38.538313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.540138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ba88-9204-4dc4-9aab-84b6596577eb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.545030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 508/1012] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:38.545383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a1774-674e-4cb4-be38-cf571898718a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:38.584587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:38.585269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:38.585550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:38.585822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:38.586611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:38.588055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f81d95-d2f7-4785-83d8-32409ddf7ca9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:38.606810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 505/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:44:39.008164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5364a80-051f-41d0-b62d-ebbb6ed23826 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.008741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5364a80-051f-41d0-b62d-ebbb6ed23826 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.008741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5364a80-051f-41d0-b62d-ebbb6ed23826 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.010261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 509/1014] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:39.015864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.016321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.016425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.034623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:39.035830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:39.054567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:39.112018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 506/1015] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:39 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:39.135705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5364a80-051f-41d0-b62d-ebbb6ed23826 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:39.241429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5364a80-051f-41d0-b62d-ebbb6ed23826 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:39.276284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:39.277208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.277486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.277774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.289481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:39.297758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:39.298700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:39.299603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:39.303981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 510/1016] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:39.304965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:39.407573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313d36e-6b92-4aa7-a965-61112937c8db None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:39.443346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:39.444122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.444406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.444682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.445436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:39.445779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:39.447189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:39.456087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 507/1017] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:39.461695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:39.486357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:39.565647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b0204-6c44-4be4-9b9f-672c2919fbf9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:39.601087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:39.601881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.602162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.602436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.612861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:39.614431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:39.631913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:39.632927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:39.633829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:39.633829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:39.633829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:39.633829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:39.636234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 511/1018] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:39.639398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:39.655901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:39.745787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f4072-0b30-40ac-84bb-a649ccf17ecf None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:39.780818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:39.781526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.781809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.782078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.782798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:39.783098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:39.784514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:39.802313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 508/1019] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:39.803730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:39.882375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:39.909720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567a402-6669-4a6a-818c-e7735f8e826a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:39.946563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:39.947367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:39.947654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:39.947967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:39.955996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:39.957461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:39.971563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:39.972523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:39.973486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:39.974623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 512/1020] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:39.978679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.053513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.086374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff86d6d-f47f-41f8-b04d-8c5cfc99ab19 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.120554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:40.121394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.121677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.121951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.122793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:40.123104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:40.124580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:40.142695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 509/1021] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:40.148235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.251844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716c82f-3624-43e1-bec2-32ae33a8e660 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.286537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:40.287191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.287474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.287747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.290501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.357964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:40.359441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:40.462595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 513/1022] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:40 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:40.468295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.567356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09f437-482f-4e2d-a859-9b01763982d9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.596132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.603470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:40.604295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.604577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.604852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.616501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:40.617974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-095b2736-0564-485d-a466-ee27a6261132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:40.635754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:40.637205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:40.638139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:40.638139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:40.638139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:40.638139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:40.639245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 510/1023] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:40.643553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.648346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defe37-83a3-4a9e-966c-d1abf5d72e24 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.648588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defe37-83a3-4a9e-966c-d1abf5d72e24 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.648862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6defe37-83a3-4a9e-966c-d1abf5d72e24 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.650394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 511/1024] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:40.655563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc37d-ce7a-429e-9997-f3f0fad9db7b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.656011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc37d-ce7a-429e-9997-f3f0fad9db7b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.656284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc37d-ce7a-429e-9997-f3f0fad9db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.673096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-115dc37d-ce7a-429e-9997-f3f0fad9db7b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:40.674279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-115dc37d-ce7a-429e-9997-f3f0fad9db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:40.682582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.747488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 512/1025] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:40.748783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa67a-4da8-44d0-ad92-bc0face57250 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.749071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa67a-4da8-44d0-ad92-bc0face57250 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.749348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aa67a-4da8-44d0-ad92-bc0face57250 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.750907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 513/1026] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:40.755904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.756186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.756463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.760879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012a8e2-fa38-42ab-a017-cb9fd7346f63 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.774079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:40.775266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:40.795889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:40.796569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.796838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.797103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.797814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:40.798116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:40.799476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:40.810443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 514/1027] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:40.811193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.846999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 514/1028] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:40.847848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.894447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.906794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:40.913450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d655302-be38-4f6b-9c79-97ee424b723f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.948122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:40.948790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:40.949070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:40.949414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:40.950127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:40.951184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f544d-97ee-408a-b404-7fce434e4947 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:40.951497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:40.967079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 515/1029] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:40 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:40.967902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:40.999297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:41.000233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:41.000486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:41.001154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:41.010990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:41.013639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:41.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:41.035692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:41.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 515/1030] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:41.042930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:41.080499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ddb8b-0bbe-4f69-965c-aa7f639d97c2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:41.120868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:41.121696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:41.122159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:41.122437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:41.123649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:41.125139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:41.127101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 516/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:41.132644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:41.159245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:41.161821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:41.165913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e853e-1bc0-4b76-9757-a46d1f1b0ee4 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:41.201564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:41.202231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:41.202506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:41.202778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:41.203559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:41.203866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:41.205341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:41.217479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 516/1032] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:41.223322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:41.251796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abadbc4-873c-4e8f-85ee-b0b6104279f8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:41.286924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:41.287426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:41.287708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:41.288086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:41.288714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:41.290148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dd965-325d-487e-95d9-a27188cb283e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:41.322497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 517/1033] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:44:41 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:41.333587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6a549-b9f4-4e06-bc66-c9fc2c374d48 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:41.372865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:41.374018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:41.374500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:41.374968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:41.376088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] RBAC: Adding query filter params (role.id=ad041f76f17d4a19aaf21217fc3fcc78, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:41.376613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:41.378339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e84d7-a058-431f-9a6d-37a0807ce077 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:41.385063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 517/1034] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:41 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=ad041f76f17d4a19aaf21217fc3fcc78 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:43.067809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3913bd-b2ba-4521-bb5a-acb89bad96fe None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.068379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3913bd-b2ba-4521-bb5a-acb89bad96fe None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.068379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3913bd-b2ba-4521-bb5a-acb89bad96fe None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.069723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 518/1035] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:43.075542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.076136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.076214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.093463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:43.094655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:43.113131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.167912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 518/1036] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:43.190509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3913bd-b2ba-4521-bb5a-acb89bad96fe None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.271411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.300994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3913bd-b2ba-4521-bb5a-acb89bad96fe None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.335074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.335894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.336188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.336465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] PATH_INFO: `/v3/services/image` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.337119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.338545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-f4baf9c0-fdbb-4bc8-bbf4-6e0edf96a1e4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:43.351295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:43.352412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:43.353250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:43.356398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 519/1037] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:43.358863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.387472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdbc-eec3-4280-a816-0936118fbff7 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.388045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdbc-eec3-4280-a816-0936118fbff7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.388463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13fdbc-eec3-4280-a816-0936118fbff7 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.389860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 520/1038] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:43.390981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c4e3f-151d-42ed-ac2d-5b9f0c7a724c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.391268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c4e3f-151d-42ed-ac2d-5b9f0c7a724c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.391551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c4e3f-151d-42ed-ac2d-5b9f0c7a724c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.392860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 521/1039] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:43.394959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274554b8-ada7-4f4f-a264-ae6fd8a7d9f8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.395238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274554b8-ada7-4f4f-a264-ae6fd8a7d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.395516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274554b8-ada7-4f4f-a264-ae6fd8a7d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.412301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-274554b8-ada7-4f4f-a264-ae6fd8a7d9f8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:43.413515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-274554b8-ada7-4f4f-a264-ae6fd8a7d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:43.461624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268790d-55c7-48be-adda-a3bd539085bc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.479641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 522/1040] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:43.485666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.485963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.486395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.496152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.496876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.497150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.497421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.498124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] RBAC: Adding query filter params (name=image) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:43.498550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.500182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.502602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:43.503795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:43.510145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 519/1041] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:43.510977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.531070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.565233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.571122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 523/1042] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:43.571858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.612638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01c84-ecfb-46f5-8876-76ddae140918 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.646751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.647436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.647788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.647989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.648776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] RBAC: Adding query filter params (type=image) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:43.649039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.650446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.655963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 520/1043] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:43.656784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.683987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e39b6-b6e9-45af-8652-ae520b32b7bc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.719301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.720098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.720379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.720655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.721837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.723290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.725128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 524/1044] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:43.725705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.736980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.757266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb26a7e-d923-4065-8a29-afd5d349fc44 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.791207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.792062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.792353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.792633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.792760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.803591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:43.811946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:43.813166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 521/1045] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:43.814223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.831897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be826bb-ad0b-4213-9403-95577a351fcb None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.868196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.868854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.869144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.869405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.870214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] RBAC: Adding query filter params (interface=public, service_id=f62a5ac9c1294a2fbcc0615c29971fd6, region_id=RegionOne) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:43.870527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.872132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.877225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 525/1046] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/endpoints?service_id=f62a5ac9c1294a2fbcc0615c29971fd6&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:43.878005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:43.918765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87968651-0011-47ed-af50-3156d6fc5760 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.952724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:43.953401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:43.953682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:43.953965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:43.954707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:43.956183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:43.966669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:43.978731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0efa3e-3bbc-42be-ac51-58ed1e0f8a60 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:43.978998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 522/1047] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:44:43 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:43.979800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.018551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:44.027080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.027942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.028230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.028508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.029288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:44.030123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.031171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.046010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 526/1048] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:44.047250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.107555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8e69-3b25-42a2-817d-81aec3e07fb0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.149645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.150316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.150593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.150867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.151551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.153092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.159985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 523/1049] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:44:43 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:44:44.167734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ae6f-482e-4d8e-b57c-27afbc5d27ef None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.205803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.206615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.206892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.207163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.220430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.221940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:44.242200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:44.243267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:44.244361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:44.244361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:44.244361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:44.244361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:44.244361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 527/1050] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:44.250354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.316945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:44.361595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e96d56-12e5-4268-adbe-9d40c272ae27 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.397887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.398561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.398839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.399109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.399834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:44.400176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.401555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.415913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 524/1051] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:44.421196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.469488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:44.527233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7924-4766-467a-8087-d7a459cdb0f6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.562270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.563065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.563344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.563614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.571350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.572807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:44.587916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:44.588935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:44.589989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 528/1052] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:44.595092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.698628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21de1-5b39-4a1b-bc57-73c8a8dd0c34 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.720216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:44.735032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.735713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.736025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.736304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.737155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:44.737469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.738900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.756604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 525/1053] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:44.762788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:44.867805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3416ae-d438-4a29-b7b2-a52695afed41 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:44.870233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:44.903985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:44.904787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:44.905070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:44.905326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:44.920104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:44.921584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:44.938806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:44.939691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:44.940999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:44.940999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:44.940999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:44.940999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:44.940999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 529/1054] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:44.946069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:45.049036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef26991-efe4-419a-a0cc-fb1f0eaec337 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:45.083626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:45.084328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:45.084612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:45.084884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:45.085605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:45.085912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:45.087292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:45.098980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 526/1055] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:45.105003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:45.207789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871ab3e-3cf6-482a-af0b-42edac2f5a23 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:45.243561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:45.244314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:45.244558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:45.244859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:45.251820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:45.253314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:45.268552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:45.269571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:45.270514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 530/1056] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:45.275979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:45.317781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:45.381709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80fe1c-1636-4e53-9c05-6f0a4f464f4c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:45.415712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:45.416404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:45.416690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:45.416969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:45.417761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:45.418138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:45.419672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed809e0-4950-42a1-9071-38ef6e324d2b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:45.432528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 527/1057] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:45.438686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:45.457182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:45.539285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2f87-c575-40b4-913c-a40e260f4ca9 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:45.573333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:45.574255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:45.574543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:45.574816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/ad041f76f17d4a19aaf21217fc3fcc78` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:45.577088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=d08cdb2d283143aa83b38bb3b04aee8c, role_id=ad041f76f17d4a19aaf21217fc3fcc78)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:45.578553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21f229-3326-4006-9733-7e1f22fdeb8b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:45.594367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 531/1058] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:45 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/d08cdb2d283143aa83b38bb3b04aee8c/roles/ad041f76f17d4a19aaf21217fc3fcc78 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:46.173140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec7f4-7c57-4a4e-9f92-17f3539dafa3 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.173488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec7f4-7c57-4a4e-9f92-17f3539dafa3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.173671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec7f4-7c57-4a4e-9f92-17f3539dafa3 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.175077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 528/1059] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:46.180598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.180736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2fa4d-9705-4e75-a0eb-83e6c9d049f6 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.180910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.181006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2fa4d-9705-4e75-a0eb-83e6c9d049f6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.181152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.181259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2fa4d-9705-4e75-a0eb-83e6c9d049f6 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.182518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 529/1060] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:46.187642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.188802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.188802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.199285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:46.201182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:46.204831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:46.206009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:46.276027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 530/1061] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:46.282612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 532/1062] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:46.299070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.308027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.317180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.331908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.408352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa98a3e-426c-4557-ac21-2bc27182c53d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.422418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95195961-1db0-49ff-8920-ccaa852401c4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.449149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.450473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.450954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.451064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] PATH_INFO: `/v3/services/image` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.451695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.453104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.459330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.460255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.460542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.460816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:46.465878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:46.466857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:46.468616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 531/1063] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:46.473178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.473710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:46.481325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:46.482270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:46.484364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 533/1064] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:46.488415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.563537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.573986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.584057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e488f67-bd78-4d07-9ea5-98ed7f4335e1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.603419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e259fb-2002-4c65-b3da-5e253f25cf6e None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.617702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.618497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.618771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.619131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.619955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] RBAC: Adding query filter params (name=image) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:46.620333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.621814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.627288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 532/1065] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:46.632779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.641140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052338e7-1fed-418a-8868-91d689665873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.641858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052338e7-1fed-418a-8868-91d689665873 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.642142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052338e7-1fed-418a-8868-91d689665873 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.642493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052338e7-1fed-418a-8868-91d689665873 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.643328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052338e7-1fed-418a-8868-91d689665873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:46.643559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052338e7-1fed-418a-8868-91d689665873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.645233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052338e7-1fed-418a-8868-91d689665873 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.659682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 534/1066] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:46.660742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052338e7-1fed-418a-8868-91d689665873 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.749261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df723367-9867-43f4-ab6f-82f87293aa93 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.770556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052338e7-1fed-418a-8868-91d689665873 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.785449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.786123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.786405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.786682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.787393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] RBAC: Adding query filter params (type=image) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:46.787707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.789120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.794299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 533/1067] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:46.799716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.800318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.809486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.810375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.810654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.810952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] PATH_INFO: `/v3/users/39b682b52f7c47888f4a587d9b8499c2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.822160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:46.827369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] RBAC: Authorizing `identity:get_user(user_id=39b682b52f7c47888f4a587d9b8499c2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.828861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.835640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 535/1068] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/users/39b682b52f7c47888f4a587d9b8499c2 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:46.836581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:46.905129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878884f7-90a5-4b6d-a4ae-e4fc4aa2b834 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.943714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463774a-31db-41a1-bdda-ca9ce87bb9ae None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:46.948233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:46.948907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:46.949235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:46.949512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] PATH_INFO: `/v3/endpoints` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:46.950356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:46.951746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:46.971599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 534/1069] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:44:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:46.999607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:47.000291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.000728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.000894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.057447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:47.079750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:47.081275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:47.192616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 536/1070] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:47.198641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:47.279233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:47.303183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaaf62-207a-4d00-9a17-16342894764b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:47.337677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:47.338547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.338802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.339095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.340353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:47.341793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ca653-4d6d-49da-971a-ff0586867a98 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:47.343725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 535/1071] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:44:47 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:47.349613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:47.432911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:47.454357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198fc9-04f0-4505-920f-c15d331d2363 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:47.491630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:47.492318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.492595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.492923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.493754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=59691d9e8da94a39a459bb7911f109e3, user.id=39b682b52f7c47888f4a587d9b8499c2) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:47.494058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:47.495504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616870b2-eeba-4b0d-820b-1ef51142859e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:47.501698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 537/1072] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:47 2022] GET /identity/v3/role_assignments?user.id=39b682b52f7c47888f4a587d9b8499c2&scope.project.id=59691d9e8da94a39a459bb7911f109e3&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:47.622999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563f2a-1684-4bde-b4da-57a41ae4f772 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.623288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563f2a-1684-4bde-b4da-57a41ae4f772 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.623721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15563f2a-1684-4bde-b4da-57a41ae4f772 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.625264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 536/1073] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:47.631124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.631464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.631677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.650096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:47.651317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:47.722801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 538/1074] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:47.746742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15563f2a-1684-4bde-b4da-57a41ae4f772 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:47.855275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15563f2a-1684-4bde-b4da-57a41ae4f772 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:47.889846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:47.890546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:47.890831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:47.891106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:47.901884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:47.910137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:47.911081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:44:47.911973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:47.916148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 537/1075] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Jul 14 10:44:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:47.917634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.000339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:48.024559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57dd0-bf7d-4910-a638-3e0f10473067 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.058620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.059297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.059579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.059850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.060628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:48.060934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.062329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:48.070604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 539/1076] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:44:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:48.076484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.080357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:48.179924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f74289-784d-4e76-9fba-ecde13643380 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.215391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.216232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.216533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.216806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.227059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.228561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:48.246543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:48.247459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:48.248917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:48.248917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:48.248917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:48.248917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:48.248917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 538/1077] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:48.253594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.354590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a09128-5909-4835-a570-38653a7f7a4f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.387962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.388618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.388898 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.389169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.389887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:48.390194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.391571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:48.398453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:48.408068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 540/1078] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:48.409920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.472241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:48.518003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1698f-0333-4e53-8973-adb2aafa2490 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.551708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.552540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.552829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.553106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] PATH_INFO: `/v3/users/nova` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.561040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.562505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:44:48.576692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:48.577681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:48.578637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 539/1079] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Jul 14 10:44:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:48.585122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.689970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d3836-12c5-4623-b724-4f0629bd74a0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.724521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.725109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.725408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.725668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.726499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:48.726823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.728324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09b31-f7a1-40bf-a104-6f36b21bed94 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:48.747123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 541/1080] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Jul 14 10:44:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:48.752867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:48.860819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e0db-313b-412e-ac8b-ee6e37a73f8e None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:48.877445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:48.896825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:48.897543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:48.897825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:48.898145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:48.977709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:48.979070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.018151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b073e-8b76-48d7-bbe2-a9e932afe617 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.018529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b073e-8b76-48d7-bbe2-a9e932afe617 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.018693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b073e-8b76-48d7-bbe2-a9e932afe617 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.020091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 542/1081] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:44:49.024931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.025209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.025479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.042122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:49.043447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:49.098482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 540/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:49.098681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.099632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.119911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 543/1083] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:44:49.120108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.139164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.207053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb247-9af7-4b0b-a18c-16d088dd4d4b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.243759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e008d71-96bd-4908-af83-53bca4cf1cf5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.248088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.248895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.249173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.249445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.260430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.262482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.279324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.280008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.280295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:49.280578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:49.281413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:49.282338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:49.282338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:49.282338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:49.282338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:49.282561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.282638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 541/1084] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:49.287317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.344083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.356648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.361477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.362960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.390757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240575ff-57a6-4365-87b2-e840c871e5d8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.428053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.429578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.430041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.430495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.431530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:49.432048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.433804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.446568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 542/1085] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:49.452196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.468014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 544/1086] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:49.473488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.557581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.565431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0df9b-83b1-46b5-8374-104c30c08ccf None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.572325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe7a30-9234-4fe9-ae50-f3feeb11fc7a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.591632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.600169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.600952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.601245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.601510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.606684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.607476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.607756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.608057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.608154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.609512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.618009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.619446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-615c9173-e90e-48a5-9d99-7347316effc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:44:49.623448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:49.624520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:49.625583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 543/1087] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:49.626233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4647-d3ab-4611-966b-d327eade44b6 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.626520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4647-d3ab-4611-966b-d327eade44b6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.626795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4647-d3ab-4611-966b-d327eade44b6 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.632272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 544/1088] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:49.632272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b4647-d3ab-4611-966b-d327eade44b6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-90559c18-21c5-492f-8453-5fa54604fdeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:44:49.641851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:49.642797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:49.643687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:44:49.643687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:44:49.643687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:44:49.643687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:44:49.643687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 545/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:44:49.644511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.644801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.645078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.662488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:49.663763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:49.733262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 546/1090] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:49.734000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.743304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b4647-d3ab-4611-966b-d327eade44b6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.778633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.779306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.779572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.779846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.780661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:49.780968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.782407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.793811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 545/1091] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:44:49.794592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.803518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.836362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14db69-8e7f-40e7-9e69-b07299ea364e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.842088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:49.871608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.872303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.872591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.872863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.873591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:49.873904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:49.875305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:49.886649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 547/1092] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:49.887943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:49.893706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df9b3-da20-40ef-9591-e82c9bcc0e89 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:49.939555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:49.940547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:49.940890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:49.941259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:49.954775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:49.962711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:49.963750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:49.964640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 546/1093] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:49.970627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:50.023929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11157487-b14d-483b-803c-323cc97dcee0 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:50.061587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:50.077657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:50.078332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:50.078612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:50.078886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:50.079726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] RBAC: Adding query filter params (role.id=ad041f76f17d4a19aaf21217fc3fcc78, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=d08cdb2d283143aa83b38bb3b04aee8c) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:50.083077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:50.083077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:50.094345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 548/1094] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/role_assignments?user.id=d08cdb2d283143aa83b38bb3b04aee8c&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=ad041f76f17d4a19aaf21217fc3fcc78 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:50.110682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b723c-bcbd-4d50-9e43-bd31021276bd None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:50.122038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:50.155650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:50.156387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:50.156672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:50.156951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:50.157734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:50.158044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:50.159493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:50.169959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 547/1095] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:50.177034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:50.302423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d81c-1eca-41ec-a916-d9a97f6b8dbf None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:50.343176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:50.344111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:50.345151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:50.345151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] PATH_INFO: `/v3/users/39b682b52f7c47888f4a587d9b8499c2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:50.347068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] RBAC: Authorizing `identity:get_user(user_id=39b682b52f7c47888f4a587d9b8499c2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:50.347612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:50.349509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 549/1096] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:44:50 2022] GET /identity/v3/users/39b682b52f7c47888f4a587d9b8499c2 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:50.355428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:50.464273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1bfba-7232-457d-8f6c-a54327ff6be1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:50.497771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:50.498637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:50.498914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:50.499180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:50.500458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:50.501893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc73817-7a86-485a-b6ae-f518c663ff31 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:50.503590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 548/1097] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:44:50 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:50.509529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:50.570743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:50.611897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79598-a4b1-4b3e-97b5-61dd3d4788e7 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:50.646374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:50.647280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:50.647566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:50.647840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3/users/39b682b52f7c47888f4a587d9b8499c2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:50.650454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=59691d9e8da94a39a459bb7911f109e3, user_id=39b682b52f7c47888f4a587d9b8499c2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:50.651942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c4b11-f4dd-4e90-839e-039da0e84bb2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:50.666329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 550/1098] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:44:50 2022] PUT /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3/users/39b682b52f7c47888f4a587d9b8499c2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:44:52.689693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e605f-d237-4c0c-b85a-f67a14908641 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:52.690270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e605f-d237-4c0c-b85a-f67a14908641 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:52.690270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e605f-d237-4c0c-b85a-f67a14908641 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:52.691624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 549/1099] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:52.697883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:52.698337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:52.698454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:52.718330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:52.719513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:52.804441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 551/1100] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:52.827711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e605f-d237-4c0c-b85a-f67a14908641 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:52.884230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:52.953115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e605f-d237-4c0c-b85a-f67a14908641 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:52.987726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:52.988541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:52.988816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:52.989087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.003326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:44:53.010925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:44:53.012038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:44:53.013367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 550/1101] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:44:53.019342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:53.071838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:53.124149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cda40e-1b50-474b-84a1-706b11e2ffef None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:53.161326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:53.161984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:53.162265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:53.162542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.163291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:53.163593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:53.165039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:53.173684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 552/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:44:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:53.180460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:53.291165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:53.296594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0e3e3-4fe0-4297-8be9-9bb9e4dd3184 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:53.337097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:53.337956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:53.338239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:53.338515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] PATH_INFO: `/v3/users/39b682b52f7c47888f4a587d9b8499c2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.339838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] RBAC: Authorizing `identity:get_user(user_id=39b682b52f7c47888f4a587d9b8499c2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:53.341483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:53.343115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 551/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:44:53 2022] GET /identity/v3/users/39b682b52f7c47888f4a587d9b8499c2 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:53.349243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:53.452211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d21ce-c265-4918-a209-fdf42990f2d2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:53.464830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:53.489109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:53.489734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:53.490010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:53.490279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.558093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:53.559507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:53.659186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 553/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:53.668485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:53.777384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151d9b4-ba6a-4025-9097-4cd12d4f7b49 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:53.814657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:53.815439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:53.815737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:53.815992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.817336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:53.818787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d9d22-fd7f-4de9-ae95-da2d38aabb99 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:53.820537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 552/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:44:53 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:53.827460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:53.846176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:53.942892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee0ce8-10f1-43ca-9c36-1ce1fe3f6461 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:53.980585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:53.981269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:53.981649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:53.981930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:53.982761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=59691d9e8da94a39a459bb7911f109e3, user.id=39b682b52f7c47888f4a587d9b8499c2) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:53.983083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:53.984594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71184e13-b98f-4fcb-afcc-6e2b3385f966 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:53.990760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 554/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:53 2022] GET /identity/v3/role_assignments?user.id=39b682b52f7c47888f4a587d9b8499c2&scope.project.id=59691d9e8da94a39a459bb7911f109e3&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:56.182545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977083a9-c12a-4833-9819-32ed77bcc36a None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:56.183156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977083a9-c12a-4833-9819-32ed77bcc36a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:56.183156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977083a9-c12a-4833-9819-32ed77bcc36a None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:56.184937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 553/1107] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:56.191585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:56.192353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:56.192455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:56.213697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:56.214887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:56.289511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 555/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:56.315181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977083a9-c12a-4833-9819-32ed77bcc36a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:56.354458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:56.424804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977083a9-c12a-4833-9819-32ed77bcc36a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:56.461152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:56.461855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:56.462131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:56.462399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:56.463613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:56.465085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986adea-8327-40b7-b576-31a480f38bdb None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:56.466774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 554/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:56.472795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:56.488964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:56.607085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9129648-7d27-4869-be40-8e8dd2c6e158 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:56.643846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:56.644613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:56.644893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:56.645199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:56.646035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:56.647482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37cf48-efcb-4528-8203-46d415e17192 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:56.672309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 556/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:44:56 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:44:58.724553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73dd3d-a67d-4038-9eaa-431d16c5506a None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:58.725149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73dd3d-a67d-4038-9eaa-431d16c5506a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:58.725149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73dd3d-a67d-4038-9eaa-431d16c5506a None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:58.726492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 555/1111] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:44:58.732087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:58.732529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:58.732632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:58.751902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:44:58.753084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:44:58.814511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:58.828606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 557/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:44:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:44:58.857349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73dd3d-a67d-4038-9eaa-431d16c5506a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:58.983090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73dd3d-a67d-4038-9eaa-431d16c5506a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:58.984597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:59.031156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:59.032041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:59.032322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:59.032592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] PATH_INFO: `/v3/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:59.034181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] RBAC: Authorizing `identity:get_role(role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:59.035562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:59.036990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 556/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:44:58 2022] GET /identity/v3/roles/a0d79cab834140ccb511ceb372f735a2 => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:59.044651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:59.151022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47826a-eed5-43c9-8dac-da4327bbb5bd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:59.188067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:59.188916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:59.189201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:59.189473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] PATH_INFO: `/v3/users/498ab81e69354b3984926ecc2a36e204` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:59.203671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] RBAC: Authorizing `identity:get_user(user_id=498ab81e69354b3984926ecc2a36e204)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:59.205180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:59.206837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 558/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:44:59 2022] GET /identity/v3/users/498ab81e69354b3984926ecc2a36e204 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:59.213033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:59.317323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b421c-58c3-494f-b29d-d9f015db5f59 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:59.352705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:59.353353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:59.353637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:59.353911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:59.398638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:59.430781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:59.432280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:59.533376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 557/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:44:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:44:59.539742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:59.593256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:59.644557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9107b7-9c19-47dd-9f02-65112d3e9b18 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:59.682442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:59.683305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:59.683587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:59.683862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:59.685417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:59.686879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd59a-bfda-4cbf-a543-2ba4e824ce9c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:59.688664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 559/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:44:59 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:44:59.695018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:44:59.771042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:44:59.803427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe8dde-4740-4392-a001-a093ec9c2caa None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:44:59.839409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:44:59.840079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:44:59.840424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:44:59.840717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:44:59.841553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=59691d9e8da94a39a459bb7911f109e3, user.id=498ab81e69354b3984926ecc2a36e204) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:44:59.841863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:44:59.843365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc717-ad8f-4ee5-abeb-42232833b9d8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:44:59.848952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 558/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:44:59 2022] GET /identity/v3/role_assignments?user.id=498ab81e69354b3984926ecc2a36e204&scope.project.id=59691d9e8da94a39a459bb7911f109e3&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:01.862838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937056-2106-40be-beea-8acc6c895c02 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:01.863435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937056-2106-40be-beea-8acc6c895c02 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:01.863435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10937056-2106-40be-beea-8acc6c895c02 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:01.864796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 560/1118] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:01.871272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:01.872062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:01.872156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:01.891972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:01.893192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:01.967261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 559/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:01.989866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10937056-2106-40be-beea-8acc6c895c02 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:02.100860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10937056-2106-40be-beea-8acc6c895c02 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:02.118292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:02.145954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:02.147302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:02.147772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:02.148307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] PATH_INFO: `/v3/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:02.150801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] RBAC: Authorizing `identity:get_role(role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:02.153194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:02.154595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 561/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:01 2022] GET /identity/v3/roles/a0d79cab834140ccb511ceb372f735a2 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:02.160648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:02.223618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:02.265674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e7a9-75f5-4d07-b7e7-ac0c44606b8f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:02.308642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:02.309554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:02.309906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:02.310184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] PATH_INFO: `/v3/users/498ab81e69354b3984926ecc2a36e204` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:02.311449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] RBAC: Authorizing `identity:get_user(user_id=498ab81e69354b3984926ecc2a36e204)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:02.312935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:02.314618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 560/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:02 2022] GET /identity/v3/users/498ab81e69354b3984926ecc2a36e204 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:02.320813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:02.429968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94af743-2a53-440d-90a1-4a156fe6af51 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:02.470358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:02.471207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:02.471489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:02.471765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:02.473107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:02.474562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8e3ab-15b4-4c0b-94c3-0d45d734878a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:02.476298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 562/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:02 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:02.483102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:02.600812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dea8c-7b81-4984-82c2-40bd5f700518 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:02.630858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:02.637170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:02.638505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:02.638791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:02.639062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3/users/498ab81e69354b3984926ecc2a36e204/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:02.641339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] RBAC: Authorizing `identity:create_grant(project_id=59691d9e8da94a39a459bb7911f109e3, user_id=498ab81e69354b3984926ecc2a36e204, role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:02.642805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338831c5-7c4c-4f95-ae5a-2dd52d99c633 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:02.658605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 561/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:45:02 2022] PUT /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3/users/498ab81e69354b3984926ecc2a36e204/roles/a0d79cab834140ccb511ceb372f735a2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:45:04.630944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a31-0265-48b1-a188-2a76b55b00c8 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:04.631522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a31-0265-48b1-a188-2a76b55b00c8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:04.631522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a31-0265-48b1-a188-2a76b55b00c8 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:04.633718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 563/1124] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:04.641402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:04.641861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:04.641940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:04.670449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:04.671641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:04.742541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 562/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:45:04.775066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02a31-0265-48b1-a188-2a76b55b00c8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:04.838795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:04.885589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02a31-0265-48b1-a188-2a76b55b00c8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:04.921400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:04.922252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:04.922534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:04.922805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] PATH_INFO: `/v3/roles/a0d79cab834140ccb511ceb372f735a2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:04.924348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] RBAC: Authorizing `identity:get_role(role_id=a0d79cab834140ccb511ceb372f735a2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:04.925739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:04.927121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 564/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:04 2022] GET /identity/v3/roles/a0d79cab834140ccb511ceb372f735a2 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:04.933901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:05.055082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854492b8-c1ba-49fa-ab30-259ba6008f0e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:05.078204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:05.090637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:05.091649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:05.091956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:05.092230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] PATH_INFO: `/v3/users/498ab81e69354b3984926ecc2a36e204` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:05.093542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] RBAC: Authorizing `identity:get_user(user_id=498ab81e69354b3984926ecc2a36e204)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:05.095003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:05.096684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 563/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:04 2022] GET /identity/v3/users/498ab81e69354b3984926ecc2a36e204 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:05.102734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:05.206678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913a17-7844-4dd7-976a-05fac99d0bed None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:05.229166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:05.241847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:05.242483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:05.242758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:05.243032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:05.310529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:05.312005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:05.417142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 565/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:05.426652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:05.542294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c2c92-6791-4968-88b7-65d02b6cd9c5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:05.578623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:05.579467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:05.579743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:05.580037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] PATH_INFO: `/v3/projects/59691d9e8da94a39a459bb7911f109e3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:05.581351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] RBAC: Authorizing `identity:get_project(project_id=59691d9e8da94a39a459bb7911f109e3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:05.582802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39956d37-a4fa-4e4a-a146-558fd0b0748b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:05.584530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 564/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:05 2022] GET /identity/v3/projects/59691d9e8da94a39a459bb7911f109e3 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:05.591448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:05.619197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:05.709015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c28f71-c30b-4b5d-8bc5-0aa035049b8a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:05.747907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:05.748558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:05.748843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:05.749100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:05.749925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] RBAC: Adding query filter params (role.id=a0d79cab834140ccb511ceb372f735a2, scope.project.id=59691d9e8da94a39a459bb7911f109e3, user.id=498ab81e69354b3984926ecc2a36e204) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:05.750249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:05.751694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66662d3-7804-479d-b293-20a97cc9b697 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:05.757661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 566/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:45:05 2022] GET /identity/v3/role_assignments?user.id=498ab81e69354b3984926ecc2a36e204&scope.project.id=59691d9e8da94a39a459bb7911f109e3&role.id=a0d79cab834140ccb511ceb372f735a2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:07.780081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d0a9-294c-457e-993c-f1b726afb5ac None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:07.780681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d0a9-294c-457e-993c-f1b726afb5ac None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:07.780681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d0a9-294c-457e-993c-f1b726afb5ac None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:07.782030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 565/1131] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:07.788365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:07.789078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:07.789220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:07.809140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:07.810574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:07.884753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 567/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:07.916035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06d0a9-294c-457e-993c-f1b726afb5ac None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:07.984778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:08.033374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06d0a9-294c-457e-993c-f1b726afb5ac None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:08.068340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:08.069121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:08.069393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:08.069658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:08.070951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:08.072420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d82f9-ac95-49f8-b6e8-14a091f614a8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:08.074027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 566/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:08.081135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:08.102969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:08.191907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0a75d-9642-4c3d-9a8c-0a6e7ccdf40a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:08.227560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:08.228232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:08.228510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:08.228780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:08.229483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:08.230925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63062cff-d993-4a52-ae88-5f69fa522d8c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:08.258891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 568/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:45:08 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:45:10.317758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065546fd-d67e-4f8c-aa3e-eda8717f7a3b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:10.318612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065546fd-d67e-4f8c-aa3e-eda8717f7a3b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:10.318612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065546fd-d67e-4f8c-aa3e-eda8717f7a3b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:10.320787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 567/1135] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:10.326878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:10.327322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:10.327425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:10.350309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:10.352211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:10.425438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:10.436829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 569/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:10.461276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065546fd-d67e-4f8c-aa3e-eda8717f7a3b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:10.583318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065546fd-d67e-4f8c-aa3e-eda8717f7a3b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:10.615947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:10.629678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:10.630481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:10.630763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:10.631033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:10.632375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:10.633818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c5294-a2c0-45a8-b1b1-e0cf54a27b0a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:10.635480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 568/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:45:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:10.641998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:10.742649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fe349-ebbf-4fb7-96d6-06828525c441 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:10.778656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:10.779287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:10.779584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:10.779852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:10.780595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:10.782025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301e079-b3e0-4498-a9ad-0bdc6927b3cc None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:10.805492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 570/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:45:10 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:45:12.861830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e5a19-d99d-4ab1-ae50-257c9327195e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:12.862755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e5a19-d99d-4ab1-ae50-257c9327195e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:12.862755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e5a19-d99d-4ab1-ae50-257c9327195e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:12.864989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 569/1139] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:12.871156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:12.871580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:12.871757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:12.898516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:12.899707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:12.954253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:12.975906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 571/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:12.999865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e5a19-d99d-4ab1-ae50-257c9327195e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:13.116474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e5a19-d99d-4ab1-ae50-257c9327195e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:13.151690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:13.152505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:13.152783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:13.153048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:13.163717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:13.171461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:13.172599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:13.178447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 570/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:45:13.179435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:13.283741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698f1c9-945e-4ebb-9e99-2efd17726dfa None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:13.319121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:13.319769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:13.320072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:13.320340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:13.321081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:13.321389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:13.322777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:13.330862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 572/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:13.336836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:13.399748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:13.445722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254735e-1aa4-4452-b196-2d6b55ddfff0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:13.480262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:13.481122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:13.481402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:13.481685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] PATH_INFO: `/v3/users/566804d8e4094adfaa88ad5b2ca924a9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:13.495703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] RBAC: Authorizing `identity:get_user(user_id=566804d8e4094adfaa88ad5b2ca924a9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:13.497199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:13.499066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 571/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:13 2022] GET /identity/v3/users/566804d8e4094adfaa88ad5b2ca924a9 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:13.505234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:13.552717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:13.610278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6aad6-4374-45b3-a139-06bb390963f1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:13.645433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:13.646064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:13.646431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:13.646698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:13.719703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:13.721115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:13.771578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:13.846664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 573/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:13.855841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:13.962988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eb62-e311-49a7-85b8-307fd46916b0 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:13.976471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:13.999669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:14.000598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:14.000883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:14.001158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] PATH_INFO: `/v3/projects/51b44501550f4dbda8655f4fd0a228df` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:14.002366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=51b44501550f4dbda8655f4fd0a228df)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:14.003827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42d87-440e-4ec8-95fb-807ccecff5e7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:14.005600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 572/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:13 2022] GET /identity/v3/projects/51b44501550f4dbda8655f4fd0a228df => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:14.012172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:14.127337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3715-f1c5-482a-bbe5-a9f677ee8c4d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:14.171389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:14.172068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:14.172342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:14.172605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:14.173421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=51b44501550f4dbda8655f4fd0a228df, user.id=566804d8e4094adfaa88ad5b2ca924a9) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:14.173725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:14.175160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5689b-4a56-49f9-b208-c714643842c0 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:14.181368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 574/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:45:14 2022] GET /identity/v3/role_assignments?user.id=566804d8e4094adfaa88ad5b2ca924a9&scope.project.id=51b44501550f4dbda8655f4fd0a228df&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:16.158162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18338-3eca-4c98-a22b-2b042d372d5f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.159000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18338-3eca-4c98-a22b-2b042d372d5f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.159000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18338-3eca-4c98-a22b-2b042d372d5f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.161451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 573/1147] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:16.167813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.168527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.168761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.192928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:16.194041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:16.196828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:16.265580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 575/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:16.288754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18338-3eca-4c98-a22b-2b042d372d5f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:16.398678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18338-3eca-4c98-a22b-2b042d372d5f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:16.432802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:16.433598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.433876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.434149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.445859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:16.453510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:16.454610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:16.455846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 574/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:45:16.465008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:16.577447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eaef1-dd4a-41f9-8657-03c587a31992 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:16.613495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:16.614149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.614431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.614707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.615447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:16.615758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:16.617174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:16.625885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 576/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:16.631959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:16.690760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:16.754331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6269-2ea0-4bcc-bfb2-5c8055e293a5 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:16.788784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:16.789630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.789903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.790167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] PATH_INFO: `/v3/users/566804d8e4094adfaa88ad5b2ca924a9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.791393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] RBAC: Authorizing `identity:get_user(user_id=566804d8e4094adfaa88ad5b2ca924a9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:16.792879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:16.794596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 575/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:16 2022] GET /identity/v3/users/566804d8e4094adfaa88ad5b2ca924a9 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:16.800578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:16.803736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:16.904050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f9b2-bc3c-4796-8cac-dd80306b4e87 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:16.938586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:16.939995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:16.939995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:16.939995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] PATH_INFO: `/v3/projects/51b44501550f4dbda8655f4fd0a228df` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:16.941214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] RBAC: Authorizing `identity:get_project(project_id=51b44501550f4dbda8655f4fd0a228df)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:16.942651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b978f-0a2a-4770-9c33-4332ba86e58b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:16.944378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 577/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:16 2022] GET /identity/v3/projects/51b44501550f4dbda8655f4fd0a228df => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:16.950525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:17.052559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b110-e1b9-41cb-9c40-b440e8295964 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:17.061321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:17.095692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:17.096621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:17.096903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:17.097174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] PATH_INFO: `/v3/projects/51b44501550f4dbda8655f4fd0a228df/users/566804d8e4094adfaa88ad5b2ca924a9/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:17.099273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] RBAC: Authorizing `identity:create_grant(project_id=51b44501550f4dbda8655f4fd0a228df, user_id=566804d8e4094adfaa88ad5b2ca924a9, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:17.100766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b2e5-d0c5-42d8-87eb-669303bbc91f None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:17.116750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 576/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:45:16 2022] PUT /identity/v3/projects/51b44501550f4dbda8655f4fd0a228df/users/566804d8e4094adfaa88ad5b2ca924a9/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:45:19.109002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a46089-b992-459b-8e6c-3d97d52b1f51 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.109645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a46089-b992-459b-8e6c-3d97d52b1f51 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.109918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a46089-b992-459b-8e6c-3d97d52b1f51 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.112371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 578/1154] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:19.120493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.121244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.121409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.149075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:19.150891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:19.224635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 577/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:19.248247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a46089-b992-459b-8e6c-3d97d52b1f51 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:19.336807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:19.381853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a46089-b992-459b-8e6c-3d97d52b1f51 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:19.417383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:19.418178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.418456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.418732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.429941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:19.437479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:19.438632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:45:19.439511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 579/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:45:19.445165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:19.547490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:19.551514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762cccd-f545-468f-b95d-3c4d1b632638 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:19.587144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:19.587792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.588103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.588372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.589111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:19.589422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:19.590782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:19.599089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 578/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:45:19.605615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:19.710362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf75d95-2c2e-4455-b9de-28eab36ba48b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:19.740374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:19.747013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:19.747931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.748198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.748477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] PATH_INFO: `/v3/users/566804d8e4094adfaa88ad5b2ca924a9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.749731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] RBAC: Authorizing `identity:get_user(user_id=566804d8e4094adfaa88ad5b2ca924a9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:19.751171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:19.752838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 580/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:19 2022] GET /identity/v3/users/566804d8e4094adfaa88ad5b2ca924a9 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:19.761530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:19.871399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f25b3-0437-44ee-aec5-d85224215fb9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:19.909430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:19.910077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:19.910355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:19.910627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:19.940468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:19.985772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:19.987260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:20.089840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 579/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:20.097093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:20.212776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2920bb4-f2cc-456c-bfae-67fa9847116c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:20.248067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:20.249019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:20.249305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:20.249580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] PATH_INFO: `/v3/projects/51b44501550f4dbda8655f4fd0a228df` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:20.250824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] RBAC: Authorizing `identity:get_project(project_id=51b44501550f4dbda8655f4fd0a228df)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:20.252308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb8560-b461-4fe3-97f2-ade0fa4755f3 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:20.254015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 581/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:20 2022] GET /identity/v3/projects/51b44501550f4dbda8655f4fd0a228df => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:20.260647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:20.322988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:20.370922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663129a3-53cb-4501-854e-f0754041a4b9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:20.406050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:20.406691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:20.406973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:20.407240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:20.408103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=51b44501550f4dbda8655f4fd0a228df, user.id=566804d8e4094adfaa88ad5b2ca924a9) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:20.408417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:20.409857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a8290-51e2-41b5-855d-5b0d83c5e611 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:20.415815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 580/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:45:20 2022] GET /identity/v3/role_assignments?user.id=566804d8e4094adfaa88ad5b2ca924a9&scope.project.id=51b44501550f4dbda8655f4fd0a228df&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:22.433440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53139c0-0da3-434f-aa3b-f04a8665e6a2 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:22.433999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53139c0-0da3-434f-aa3b-f04a8665e6a2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:22.433999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53139c0-0da3-434f-aa3b-f04a8665e6a2 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:22.435357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 582/1162] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:22.442015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:22.442702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:22.442899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:22.463607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:22.464856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:22.539617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 581/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:22.562313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53139c0-0da3-434f-aa3b-f04a8665e6a2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:22.631837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:22.669175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53139c0-0da3-434f-aa3b-f04a8665e6a2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:22.705067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:22.705643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:22.705923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:22.706195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:22.775179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:22.776660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:22.841078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:22.877475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 583/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:22.886627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:22.943791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:22.990520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d528978-b012-4cbb-8d0b-48f5c26c5861 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:23.029676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:23.030464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:23.030727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:23.031017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:23.037879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:23.039331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-bfe72983-df1f-43f3-964b-a18d63a20926 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:23.053497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:23.054649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:23.055967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 582/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:45:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:45:23.061082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:23.173186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba5e01-f54a-468f-b825-59580cc2279a None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:23.209514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:23.210169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:23.210448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:23.210722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:23.211443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:23.211751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:23.213169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b87f45-25ba-436f-9c96-a24c897d22ce None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:23.221142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 584/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Jul 14 10:45:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:25.256668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bed2d-4cbe-4cc0-aa12-7852d97dcf11 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:25.257475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bed2d-4cbe-4cc0-aa12-7852d97dcf11 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:25.257475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bed2d-4cbe-4cc0-aa12-7852d97dcf11 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:25.259627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 583/1167] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:25.269127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13b50-f0ff-4a53-b815-e1cf96fc87d3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:25.269815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13b50-f0ff-4a53-b815-e1cf96fc87d3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:25.270026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab13b50-f0ff-4a53-b815-e1cf96fc87d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:25.292753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cab13b50-f0ff-4a53-b815-e1cf96fc87d3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:25.293939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cab13b50-f0ff-4a53-b815-e1cf96fc87d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:25.320002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:25.367080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 585/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:25.390526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bed2d-4cbe-4cc0-aa12-7852d97dcf11 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:25.496707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bed2d-4cbe-4cc0-aa12-7852d97dcf11 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:25.511055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:25.547631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:25.548364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:25.548644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:25.548946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:25.549702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:25.551091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a60050-70ed-41c4-91cd-db4725b1bb9e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:25.574066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 584/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Thu Jul 14 10:45:25 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:45:27.621330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30389c31-a206-4546-9863-d1be06c5ccd9 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:27.622343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30389c31-a206-4546-9863-d1be06c5ccd9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:27.622343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30389c31-a206-4546-9863-d1be06c5ccd9 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:27.624345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 586/1170] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:27.630549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:27.631246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:27.631348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:27.657694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:27.658874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:27.730203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 585/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:27.767570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30389c31-a206-4546-9863-d1be06c5ccd9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:27.874125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30389c31-a206-4546-9863-d1be06c5ccd9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:27.909558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:27.910406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:27.910684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:27.910954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] PATH_INFO: `/v3/domains/1439bb9829c64e3ab939a33d0d785a8e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:27.921992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=1439bb9829c64e3ab939a33d0d785a8e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:27.923453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd10596-bd10-4487-bbee-92896a711e1e None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:27.925162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 587/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Thu Jul 14 10:45:27 2022] GET /identity/v3/domains/1439bb9829c64e3ab939a33d0d785a8e => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:27.931249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:28.046451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1cfec-17a7-4f30-930a-8524369fbcae None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:28.048779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:28.091406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:28.092086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:28.092404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:28.092680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:28.093383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:28.094829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ab9be-33eb-4a49-a91f-678c7ef6a3f7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:28.126916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 586/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:45:27 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:45:30.206812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff941c-c75c-494f-8d01-f60e74fd6a04 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:30.207702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff941c-c75c-494f-8d01-f60e74fd6a04 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:30.207702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff941c-c75c-494f-8d01-f60e74fd6a04 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:30.209764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 588/1174] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:30.217795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:30.218253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:30.218359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:30.237710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:30.238888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:30.313009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 587/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:45:30.350041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff941c-c75c-494f-8d01-f60e74fd6a04 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:30.373733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:30.464289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff941c-c75c-494f-8d01-f60e74fd6a04 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:30.498505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:30.499615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:30.499906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:30.500192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] PATH_INFO: `/v3/domains/1439bb9829c64e3ab939a33d0d785a8e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:30.501416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1439bb9829c64e3ab939a33d0d785a8e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:30.502856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4243-6926-4634-ad57-b14231c535e1 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:30.504477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 589/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Thu Jul 14 10:45:30 2022] GET /identity/v3/domains/1439bb9829c64e3ab939a33d0d785a8e => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:30.513806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:30.606811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:30.624146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975591d-21d5-476f-913c-d603d01bcc50 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:30.658207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:30.658861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:30.659145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:30.659417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:30.660134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:30.661569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbd74b-c2b4-4c92-9cf1-691a2c02fb9a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:30.683209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 588/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:45:30 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:45:32.755128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b6fd-7b3b-42b8-911b-0515476be105 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:32.756022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b6fd-7b3b-42b8-911b-0515476be105 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:32.756022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b6fd-7b3b-42b8-911b-0515476be105 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:32.758252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 590/1178] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:32.764618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:32.765573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:32.765573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:32.788370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:32.789555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:32.859969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 589/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:32.889036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b6fd-7b3b-42b8-911b-0515476be105 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:32.991207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:33.004631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b6fd-7b3b-42b8-911b-0515476be105 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:33.041068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:33.041801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:33.042079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:33.042349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:33.055715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:33.063980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:33.065102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:45:33.066076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 591/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:45:33.071651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:33.146779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:33.186063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a36d76-a1df-45f6-84a2-5d67296623f4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:33.224290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:33.224946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:33.225220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:33.225486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:33.226226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:33.226526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:33.227913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:33.238751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 590/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:33.248088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:33.360792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22d257-ca01-49d6-b070-0b6bc1f77e92 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:33.398426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:33.399292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:33.399571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:33.399847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] PATH_INFO: `/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:33.413401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9c8b0782e4c99b86a3171ab0e3d84)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:33.414860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:33.416545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 592/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:33 2022] GET /identity/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:33.423302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:33.538146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc6baa-7b3b-429d-a8bb-652ba49a7cb7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:33.573202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:33.573847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:33.574123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:33.574398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:33.577712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:33.644750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:33.646147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:33.747180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 591/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:33.753581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:33.763627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:33.877027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43462-1633-4453-a18f-4384a61e8037 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:33.913418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:33.914188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:33.914462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:33.914731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] PATH_INFO: `/v3/projects/881fe9f3236242eeba460894d9b88d39` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:33.916018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] RBAC: Authorizing `identity:get_project(project_id=881fe9f3236242eeba460894d9b88d39)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:33.917453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b873d2a-dc11-452b-8b36-8a2d0bcf910c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:33.919143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 593/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:33 2022] GET /identity/v3/projects/881fe9f3236242eeba460894d9b88d39 => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:33.925798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:33.962210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:34.030542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0793-4acd-4577-a99a-d59f510b40a3 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:34.070097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:34.070736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:34.071011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:34.071279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:34.072130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=881fe9f3236242eeba460894d9b88d39, user.id=05d9c8b0782e4c99b86a3171ab0e3d84) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:34.072443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:34.073855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e015-cd1d-49ab-87a2-1270f622d15e None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:34.079462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 592/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:45:33 2022] GET /identity/v3/role_assignments?user.id=05d9c8b0782e4c99b86a3171ab0e3d84&scope.project.id=881fe9f3236242eeba460894d9b88d39&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:36.070863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a7e7-cdd8-4e20-b2a2-bb24ad7d8264 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.071447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a7e7-cdd8-4e20-b2a2-bb24ad7d8264 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.071447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a7e7-cdd8-4e20-b2a2-bb24ad7d8264 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.072840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 594/1186] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:36.079356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.080146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.080279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.109282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:36.109703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:36.179646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 593/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:36.203177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4a7e7-cdd8-4e20-b2a2-bb24ad7d8264 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:36.284302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:36.318394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4a7e7-cdd8-4e20-b2a2-bb24ad7d8264 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:36.361196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:36.362344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.362723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.363095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.378225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:36.385725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:36.386825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:45:36.387862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 595/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:45:36.395915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:36.424045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:36.494075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892dd5c-9850-4be7-bd68-3b8debc3266a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:36.530020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:36.530842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.531122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.531399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.532178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:36.532487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:36.533875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:36.542055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 594/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:36.548168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:36.658424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd88ed-1ec1-4303-8880-c788ac524fbc None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:36.693815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:36.694680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.694957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.695228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] PATH_INFO: `/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.696566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9c8b0782e4c99b86a3171ab0e3d84)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:36.698020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:36.699665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 596/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:36 2022] GET /identity/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:36.706582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:36.819936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:36.830902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c24ef-d3ac-4d8a-8119-482674d0ff21 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:36.870131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:36.870996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:36.871275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:36.871547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] PATH_INFO: `/v3/projects/881fe9f3236242eeba460894d9b88d39` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:36.872900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] RBAC: Authorizing `identity:get_project(project_id=881fe9f3236242eeba460894d9b88d39)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:36.874450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f05a6-0efa-4f62-b807-400567bde094 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:36.876205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 595/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:36 2022] GET /identity/v3/projects/881fe9f3236242eeba460894d9b88d39 => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:36.885273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:36.888601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:36.989668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a184af-0f1d-4dad-9c35-ea90f62a8361 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:37.024912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:37.025799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:37.026081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:37.026358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] PATH_INFO: `/v3/projects/881fe9f3236242eeba460894d9b88d39/users/05d9c8b0782e4c99b86a3171ab0e3d84/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:37.028380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=881fe9f3236242eeba460894d9b88d39, user_id=05d9c8b0782e4c99b86a3171ab0e3d84, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:37.029842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4fff-e54c-49ad-b923-964dad10f1b5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:37.046681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 597/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:45:36 2022] PUT /identity/v3/projects/881fe9f3236242eeba460894d9b88d39/users/05d9c8b0782e4c99b86a3171ab0e3d84/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:45:39.057077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a64e-cfab-4df9-be50-f633426b6a34 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.057955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a64e-cfab-4df9-be50-f633426b6a34 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.057955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a64e-cfab-4df9-be50-f633426b6a34 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.060242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 596/1193] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:45:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:45:39.066499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.067243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.067391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.088057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:45:39.089241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:45:39.147462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:39.159853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 598/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:45:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:45:39.184112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34a64e-cfab-4df9-be50-f633426b6a34 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:39.296441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34a64e-cfab-4df9-be50-f633426b6a34 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:39.333697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:39.334486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.334768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.335037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.346848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:45:39.354451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:45:39.355536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:45:39.357188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 597/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:45:39.364222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:39.468629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b96ee-64bb-4089-8234-826c7e701a57 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:39.503852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:39.504529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.504810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.505082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.505826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:39.506136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:39.507516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:39.516218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 599/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:45:39.522779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:39.546159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:39.631589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c6c36-5fda-4199-99d3-d9785990b558 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:39.668588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:39.669479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.669759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.670038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] PATH_INFO: `/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.671401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9c8b0782e4c99b86a3171ab0e3d84)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:39.672897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:39.674531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 598/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Jul 14 10:45:39 2022] GET /identity/v3/users/05d9c8b0782e4c99b86a3171ab0e3d84 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:39.683042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:39.743356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:39.794829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6dd1a-e659-4d1d-8a43-4ad4ecd37fa6 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:39.832608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:39.833245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:39.833522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:39.833789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:39.914288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:39.915673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:39.970434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:40.017406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 600/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:45:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:45:40.023694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:40.115124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:40.133985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10875-524a-4150-a858-dd066335cf73 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:40.171391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:40.172293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:40.172572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:40.172840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] PATH_INFO: `/v3/projects/881fe9f3236242eeba460894d9b88d39` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:40.174019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] RBAC: Authorizing `identity:get_project(project_id=881fe9f3236242eeba460894d9b88d39)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:40.175457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82adb3-f527-44ac-9636-1502860c07d2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:40.177185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 599/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Jul 14 10:45:40 2022] GET /identity/v3/projects/881fe9f3236242eeba460894d9b88d39 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:40.186315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:45:40.300984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142c2-d2d7-4dfb-b116-cd7640a72c45 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:45:40.336247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:45:40.336907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:45:40.337162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:45:40.337467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:45:40.338472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=881fe9f3236242eeba460894d9b88d39, user.id=05d9c8b0782e4c99b86a3171ab0e3d84) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:45:40.338791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:45:40.340276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495363c-ed1c-424a-85b8-f209130162cb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:45:40.346487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 601/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:45:40 2022] GET /identity/v3/role_assignments?user.id=05d9c8b0782e4c99b86a3171ab0e3d84&scope.project.id=881fe9f3236242eeba460894d9b88d39&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:45:50.167242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=23 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:45:50.344186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:36.338682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb71b-2d27-4071-8c4a-3f776a31810e None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:36.339238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb71b-2d27-4071-8c4a-3f776a31810e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:36.339238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb71b-2d27-4071-8c4a-3f776a31810e None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:36.340557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 600/1201] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:36.346384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:36.346926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:36.346926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:36.370364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:36.371578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:36.468055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 602/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:46:36.487964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facb71b-2d27-4071-8c4a-3f776a31810e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:36.620382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facb71b-2d27-4071-8c4a-3f776a31810e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:36.664407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:36.665431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:36.665703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:36.666097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:36.681163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:36.682895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:36.705278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:36.708882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:36.709866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:36.709866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:36.709866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:36.709866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:36.709866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 601/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:46:36.716366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:36.853071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511641b-dcf2-4a03-82ee-e6a4e4db0880 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:36.886141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:36.896157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:36.896925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:36.897183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:36.897494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:36.898335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:36.898658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:36.900217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94f1c0-cc69-4b7a-be0c-c16dd0180994 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:36.913630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 603/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:36.920204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:36.998067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:37.057091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6470f-a90e-4ac2-b952-5a0f40bb616c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:37.107143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:37.107853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:37.108160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:37.108493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:37.109310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:37.111142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840e6c3-8541-48a3-8be2-588007b78b45 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:37.135049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 602/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Jul 14 10:46:36 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:46:39.496052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff686-77bb-4882-94d1-15ece4736c9e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:39.496658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff686-77bb-4882-94d1-15ece4736c9e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:39.496772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff686-77bb-4882-94d1-15ece4736c9e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:39.498095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 604/1206] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:39.506234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:39.506234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:39.506234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:39.524537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:39.525780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:39.608579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 603/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:46:39.633686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff686-77bb-4882-94d1-15ece4736c9e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:39.753127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff686-77bb-4882-94d1-15ece4736c9e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:39.796975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:39.797863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:39.798090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:39.798486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:39.811524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:46:39.820771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:39.824014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:39.824923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 605/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:46:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:39.829187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:39.874649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:39.949749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27605f-5b88-488f-a0f3-45abb2f7ce85 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:39.986862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:39.987534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:39.987810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:39.988111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:39.988908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:39.989210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:39.990639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:39.999630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 604/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:40.005882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:40.065861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:40.128736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a5958-baba-4ba7-addc-145381912152 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:40.172173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:40.173135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:40.173772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:40.174191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:40.186850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:40.188442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:40.210370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:40.211689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:40.217042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:40.217042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:40.217042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:40.217042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:40.217042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 606/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:40.217674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:40.333064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:40.336347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a145fe5-2583-4e0a-bc6b-bc5543320fa6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:40.378621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:40.379342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:40.379628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:40.379929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:40.380775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:40.381075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:40.382520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:40.400252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 605/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:40.401565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:40.530330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579e06c-711f-474c-893f-77c7157a2359 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:40.532037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:40.568352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:40.569261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:40.569612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:40.569936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] PATH_INFO: `/v3/users/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:40.578558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:40.580724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:46:40.595086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:40.596251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:40.597378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 607/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:40.602887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:40.714081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26959fa-f636-48d6-b1c3-571792478cbb None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:40.757535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:40.759356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:40.759356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:40.759356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:40.759630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:40.759913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:40.761351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:40.781456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 606/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:46:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:40.788069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:40.908495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e03be-ea4d-4b40-88c6-db72e3fa5acf None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:40.955933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:40.956696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:40.957290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:40.957290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:40.995208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:41.042062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:41.043921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:41.166152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 608/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:46:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:46:41.174886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:41.200381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:41.297706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4345827-7712-4e46-b0b5-dc7b1677ca52 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:41.344333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:41.345203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:41.346115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:41.346371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:41.357387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:41.358984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:41.379357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:41.380585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:41.381511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:41.381511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:41.381511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:41.381511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:41.381511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 607/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:41.386789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:41.466348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:41.510932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc9627-572b-48b8-bb79-53081980889f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:41.550105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:41.550798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:41.551037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:41.551310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:41.552070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:41.552370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:41.554841 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:41.567294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 609/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:41.574132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:41.653365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:41.691594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cdbca-f671-4d16-a526-9ce6d739605b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:41.727739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:41.728496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:41.728771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:41.729053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:41.737774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:41.739339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:46:41.756657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:41.757969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:41.759314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 608/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:41.764182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:41.876022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5ca6-19da-4757-ad84-499e7cec3d0c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:41.903522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:41.921287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:41.921959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:41.922235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:41.922504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:41.923421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:41.923733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:41.925230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523084f-fe87-482f-95e9-590590a28e8f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:41.943440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 610/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:46:41.949835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:42.071768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa88e4-1999-4ce1-bdd7-0dca7e2bfe72 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:42.101963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:42.108489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:42.109190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:42.110290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:42.110290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:42.110584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=17f4edebe2ed45a08ef7460d812c563d) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:42.110955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:42.113135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490af7f-751d-4eda-8e0d-7eb3cae80fb1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:42.119975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 609/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:46:41 2022] GET /identity/v3/role_assignments?user.id=17f4edebe2ed45a08ef7460d812c563d&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:44.224120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cce90-e8ca-4683-bb3b-c7ae890a37a0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:44.224701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cce90-e8ca-4683-bb3b-c7ae890a37a0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:44.224701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cce90-e8ca-4683-bb3b-c7ae890a37a0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:44.226071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 611/1220] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:44.232147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:44.232621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:44.232621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:44.258623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:44.260176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:44.355913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 610/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:46:44.370025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cce90-e8ca-4683-bb3b-c7ae890a37a0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:44.481032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cce90-e8ca-4683-bb3b-c7ae890a37a0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:44.517071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:44.517910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:44.518187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:44.518462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:44.529118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:46:44.537401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:44.538425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:44.539288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 612/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:46:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:44.545103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:44.673043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f59e9c-cf05-42db-9820-38d197a13b7c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:44.698665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:44.712211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:44.713262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:44.713551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:44.714094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:44.714860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] RBAC: Adding query filter params (name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:44.715172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:44.716829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:44.728704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 611/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:44.734499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:44.785213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:44.849766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a434e33-ab7e-406c-a039-b2f119ed7b7f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:44.887648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:44.888715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:44.889065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:44.889404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:44.902597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:44.904370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:44.924996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:44.926849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:44.928054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:44.928054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:44.928054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:44.928054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:44.928054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 613/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:44.933034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.042110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96523b-c92b-463a-b431-27775dbe2ee2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.076558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.077218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.077499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.077768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.078484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:45.078788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.080259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.097748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 612/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:45.099335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.200167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:45.207448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21d793-ac67-4fcf-9e10-7c6890062644 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.243571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.244414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.244693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.244967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] PATH_INFO: `/v3/users/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.254464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.256133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:46:45.272273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:45.273579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:45.274793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 614/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:45.280033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.331667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:45.388485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdc050-3300-49f6-a428-52c4161467b2 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.423693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.424837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.425117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.425390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.426221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:45.426525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.427985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.445593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 613/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:45.451483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.559104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87ff9f-fe80-40f6-97af-6590fb3181af None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.592688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.593395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.593630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.593932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.605054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.606502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:45.624550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:45.625659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:45.626808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:45.626808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:45.626808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:45.626808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:45.626808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 615/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:45.632264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.740265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3c080-3e7b-4ad4-a94d-0df218677e13 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.742806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:45.777055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.777735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.778017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.778294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.779024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:45.779329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.780746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.793828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 614/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:45.799804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:45.810073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:45.910480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311246b1-57a1-446f-bfbe-1a9d167a6e44 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:45.949163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:45.950021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:45.950305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:45.950629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:45.959470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:45.960972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:46:45.976158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:45.977285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:45.978290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:45.979469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 616/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:45.983507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:46.099275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37986c68-05e5-4d61-b179-f3cf2e2ddb49 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:46.137556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:46.138255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:46.138543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:46.138812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:46.139610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:46.139949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:46.141527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85665c09-4f80-4bea-9061-4d1dd187a036 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:46.154864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 615/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:46.161251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:46.230894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:46.276634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde78388-1d93-46d8-9a59-54da5449c45f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:46.316854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:46.317754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:46.318030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:46.318339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/17f4edebe2ed45a08ef7460d812c563d/roles/9f8ac3df182944b2bf9b4c771d4e0c47` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:46.334039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=17f4edebe2ed45a08ef7460d812c563d, role_id=9f8ac3df182944b2bf9b4c771d4e0c47)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:46.335613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c3f41-a13c-4b42-bb55-1a0410530877 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:46.349746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 617/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:46:46 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/17f4edebe2ed45a08ef7460d812c563d/roles/9f8ac3df182944b2bf9b4c771d4e0c47 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:46:48.422361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f31a-40f7-4ecf-ab3f-78f0b35855d7 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:48.422958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f31a-40f7-4ecf-ab3f-78f0b35855d7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:48.422958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1f31a-40f7-4ecf-ab3f-78f0b35855d7 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:48.424470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 616/1233] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:48.430270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:48.430751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:48.430904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:48.447961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:48.449267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:48.525221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 618/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:46:48.550704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f31a-40f7-4ecf-ab3f-78f0b35855d7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:48.579128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:48.667471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1f31a-40f7-4ecf-ab3f-78f0b35855d7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:48.727319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:48.728588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:48.729073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:48.729548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:48.746702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:46:48.755920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:48.757045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:46:48.758020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:48.758816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 617/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Jul 14 10:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:48.764837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:48.814807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:48.872179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea40c8-eaf0-4655-9a34-4066bc4db016 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:48.906299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:48.906969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:48.907252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:48.907575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:48.908635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] RBAC: Adding query filter params (name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:48.908890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:48.910279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:48.919946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 619/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Jul 14 10:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:48.925732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.014089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:49.031513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f5dfc-57ac-48a6-85f1-4992e8cf1fe7 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:49.066220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:49.067020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:49.067323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:49.067599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:49.079231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:49.080829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:49.098249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:49.099342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:49.100532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:49.100532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:49.100532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:49.100532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:49.104237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 618/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:49.105327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.204022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9aaf39-98f7-4ed1-b730-43629ef18008 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:49.213590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:49.238747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:49.239416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:49.239703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:49.240010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:49.240737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:49.241042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:49.242438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:49.253901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 620/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:49.260284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.366563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd000f-11ab-4539-bd71-fa3ab5a8f6a9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:49.402967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:49.403959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:49.404330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:49.404690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:49.407666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:49.413892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:49.415884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:46:49.433008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:49.434684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:49.435991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 619/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:49.440477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.550866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394078b-c2f7-4e3d-acd6-173130193b58 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:49.585733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:49.586408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:49.586682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:49.586952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:49.587817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:49.588148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:49.589584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:49.607240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 621/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:46:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:46:49.613353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.718463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556abd-ee8b-43d4-898f-523664f257b1 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:49.752392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:49.753064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:49.753343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:49.753616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:49.816680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:49.823649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:49.825127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:49.929122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 620/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:46:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:46:49.935195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:49.963730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:50.040354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1c111-9b43-4551-9955-319df061f991 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:50.078661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:50.079467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:50.079769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:50.080080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:50.091020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:50.092593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:50.110343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:50.111395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:50.112378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:50.112378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:50.112378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:50.112378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:50.116406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 622/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:50.117555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:50.215491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:50.222276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164e0c6-6449-4947-abb9-149f823dfa46 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:50.259212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:50.259905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:50.260256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:50.260534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:50.261254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:50.261554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:50.263100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:50.280063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 621/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:50.282761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:50.367189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:50.391067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a7807-1c32-4d92-a118-925ad8ec81a2 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:50.430886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:50.432029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:50.432327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:50.432740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:50.442852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:50.444874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:46:50.463070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:50.464297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:50.465256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 623/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:50.472346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:50.580424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8aa5c-a560-4da4-856f-52a2411bf8f6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:50.615796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:50.616505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:50.616766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:50.617038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:50.617873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:50.618127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:50.619570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefba97d-35f4-460f-953e-2e6ecfc5f3fb None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:50.632800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 622/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:50.639921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:50.772065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2e1f8-e045-499a-bfed-b6b2b0acee06 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:50.822049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:50.822995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:50.823127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:50.823376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:50.824247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] RBAC: Adding query filter params (role.id=9f8ac3df182944b2bf9b4c771d4e0c47, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=17f4edebe2ed45a08ef7460d812c563d) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:50.824557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:50.826007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad845ef-36a3-41bc-9fc4-aab39047c17f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:50.835204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 624/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:46:50 2022] GET /identity/v3/role_assignments?user.id=17f4edebe2ed45a08ef7460d812c563d&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=9f8ac3df182944b2bf9b4c771d4e0c47 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:53.079769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b9cb2-cd2c-4099-9df5-29bfe25b5805 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.080583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b9cb2-cd2c-4099-9df5-29bfe25b5805 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.080583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b9cb2-cd2c-4099-9df5-29bfe25b5805 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.082097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 623/1247] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:53.087340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.087769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.087923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.093193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:53.108135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:53.109402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:53.185269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 625/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:46:53.210274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b9cb2-cd2c-4099-9df5-29bfe25b5805 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:53.229962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:53.323198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b9cb2-cd2c-4099-9df5-29bfe25b5805 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:53.363010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:53.364037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.364514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.364878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.377791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:46:53.386366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:53.387490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:53.388557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 624/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:53.394824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:53.509653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cda035-313e-4b49-a6a5-d2d2ae6139e0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:53.540343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:53.546528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:53.547276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.547560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.547901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.548708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:53.549309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:53.550741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:53.559060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 626/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:53.565501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:53.655011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:53.684872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0113311-fc1f-4640-9f52-52e8c00d0d6d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:53.726705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:53.727597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.728043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.728386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.740693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:53.742234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:53.759970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:53.761170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:53.762449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:53.762449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:53.762449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:53.762449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:53.762449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 625/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:53.768056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:53.880588 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb995c-6405-4ad3-a5cf-844530259896 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:53.924807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:53.925489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:53.925767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:53.926045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:53.926775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:53.927085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:53.928552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:53.942002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 627/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:53.948125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:54.064957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2eb6f-a6de-4a86-a547-4707dcc8e225 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:54.093518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:54.102899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:54.103708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:54.104043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:54.104297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] PATH_INFO: `/v3/users/placement` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:54.112584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:54.114114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:46:54.130064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:54.131318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:54.132360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 626/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:54.137821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:54.199357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:54.252471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9332a43-cb37-4ce1-91b6-6c6ac097031f None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:54.292820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:54.293692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:54.293959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:54.294234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:54.295100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:54.295402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:54.297006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:54.318918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 628/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:46:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:54.324739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:54.435546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401695ae-feeb-4c27-aa9c-ff2d5b0dc204 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:54.474460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:54.475121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:54.475448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:54.475746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:54.544912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:54.559045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:54.560622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:54.673595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 627/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:46:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:46:54.680145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:54.796531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaee7e-fee4-4d41-ab96-a2c0893dc2a4 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:54.812217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:54.836712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:54.837515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:54.837794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:54.838064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:54.849691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:54.851482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:54.874580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:54.875739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:54.876978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:54.876978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:54.876978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:54.876978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:54.876978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 629/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:54.881716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:54.987857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:55.005559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d2768-b7b9-4a04-9f41-625d111f6aa9 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:55.051328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:55.052043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:55.052340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:55.052617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:55.053352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:55.053734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:55.055168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:55.068771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 628/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:55.074539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:55.188095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22b50-9b0a-424c-8640-6ad66b5577f1 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:55.226629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:55.227606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:55.227905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:55.228205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] PATH_INFO: `/v3/projects/service` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:55.236258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:55.238058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:46:55.252985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:55.254126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:46:55.255088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 630/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:55.260601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:55.374528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c1c42-a4f8-4487-bda0-518471a87acf None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:55.415315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:55.416019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:55.416302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:55.416574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:55.417359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:55.417676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:55.419107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea430-b7f0-4d58-af44-393c20a417cc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:55.431576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 629/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:46:55.437968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:55.465971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:55.562933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0a89-a7d1-498a-9790-0902e526520c None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:55.600221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:55.600981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:55.601262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:55.601553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:55.602388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=17f4edebe2ed45a08ef7460d812c563d) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:55.602751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:55.604258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a65c-8bdc-4eb8-8f0c-11bc6a788d3c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:55.610473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 631/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:46:55 2022] GET /identity/v3/role_assignments?user.id=17f4edebe2ed45a08ef7460d812c563d&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:57.635297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171c-24b5-4af5-bf4a-8e22192629e4 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:57.635993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171c-24b5-4af5-bf4a-8e22192629e4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:57.635993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9171c-24b5-4af5-bf4a-8e22192629e4 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:57.637258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 630/1261] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:46:57.643217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:57.643740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:57.643740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:57.670277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:46:57.672542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:46:57.778963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 632/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:46:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:46:57.807439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9171c-24b5-4af5-bf4a-8e22192629e4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:57.820720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:57.933185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9171c-24b5-4af5-bf4a-8e22192629e4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:57.993770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:57.994575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:57.994856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:57.995125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.006720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:46:58.014495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:58.015659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:58.016830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 631/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:58.023216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:58.046924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:58.152687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1cfd-d53f-4d69-ac5f-336ce2ea591b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:58.193941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:58.195824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:58.196140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:58.196414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.197335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:58.197635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:58.199138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:58.209389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 633/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:58.216291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:58.306567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:58.346599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a3cb-c65c-4c8b-a87a-edff915b1cf6 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:58.383801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:58.384626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:58.384900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:58.385168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.396838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:58.398295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:58.415112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:58.416114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:58.417179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:58.417179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:58.417179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:58.417179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:58.417179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 632/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:58.424661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:58.515271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:58.544447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffddb0-bbc0-4cf2-a142-5fc540578f1d None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:58.584013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:58.584678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:58.584955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:58.585226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.585949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:58.586256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:58.587646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:58.600528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 634/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:58.607627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:58.722835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c9446-1237-4b2a-9c70-ef7855e5701d None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:58.746035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:58.760935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:58.761736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:58.762017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:58.762283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] PATH_INFO: `/v3/users/placement` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.770069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:58.771814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:46:58.786145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:58.787221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:58.788249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 633/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:46:58.795806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:58.904589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b5bdd-45e5-419a-adb3-81c58e2e75da None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:58.939234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:58.939908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:58.940189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:58.940449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:58.941478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:58.941785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:58.943220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:58.951997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:58.961763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 635/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:58.967798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:59.084225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e93ae-66be-4ebc-93bf-85d67caf343a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:59.131265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:59.132077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:59.132353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:59.132627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:59.148480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:59.149955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:46:59.168133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:59.169162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:59.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:46:59.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:46:59.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:46:59.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:59.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 634/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:59.175813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:59.289402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30e6c6-79cb-4a83-b522-d1827dc51beb None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:59.335788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:59.336509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:59.336785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:59.337054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:59.337754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:59.338058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:59.339444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:59.350797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 636/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:59.356842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:59.391040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:59.467539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a42e11-f050-4940-a369-64c337712d1a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:59.506257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:59.507037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:59.507318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:59.507587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:59.514482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:59.515931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:46:59.531139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:46:59.532177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:46:59.533266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 635/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:46:59.539542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:59.610357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:59.656753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe4e49-24e9-4b22-b57c-78c63583a963 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:59.692734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:59.693398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:59.693683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:59.693951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:59.694753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:46:59.695044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:59.696532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a360f2-c5f8-4c13-a64e-54585c05c2b4 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:59.709344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 637/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:46:59.718812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:46:59.831246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:46:59.841018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ead9c-e43e-47fd-b3d0-9a4bcedb1c49 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:46:59.879779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:46:59.880624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:46:59.880899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:46:59.881168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] PATH_INFO: `/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/17f4edebe2ed45a08ef7460d812c563d/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:46:59.883371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] RBAC: Authorizing `identity:create_grant(project_id=b49330c5b3874cde8c864bc5b1a98a69, user_id=17f4edebe2ed45a08ef7460d812c563d, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:46:59.884831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a7f9f-8f0d-4b68-974d-0dfec1966241 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:46:59.901716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 636/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Jul 14 10:46:59 2022] PUT /identity/v3/projects/b49330c5b3874cde8c864bc5b1a98a69/users/17f4edebe2ed45a08ef7460d812c563d/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:47:01.894274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50d02-ade9-4981-ae43-dcfd763aa4f8 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:01.895470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50d02-ade9-4981-ae43-dcfd763aa4f8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:01.895470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d50d02-ade9-4981-ae43-dcfd763aa4f8 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:01.897757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 638/1274] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:47:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:47:01.906747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:01.907181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:01.907308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:01.926675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:47:01.927862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:47:02.002334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 637/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:47:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:47:02.026395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50d02-ade9-4981-ae43-dcfd763aa4f8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.133055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d50d02-ade9-4981-ae43-dcfd763aa4f8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:02.150931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:02.169704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:02.170489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:02.170772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:02.171047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:02.182204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:47:02.189827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:02.190979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:02.192586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 639/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:02.199249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.279594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:02.299536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49ef67-b7ad-46a0-8e96-78af66bf83b8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:02.334581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082892-69dc-465f-b406-756e8d658457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:02.335227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082892-69dc-465f-b406-756e8d658457 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:02.335508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082892-69dc-465f-b406-756e8d658457 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:02.335780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24082892-69dc-465f-b406-756e8d658457 None admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:02.336550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24082892-69dc-465f-b406-756e8d658457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:02.336852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24082892-69dc-465f-b406-756e8d658457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:02.338218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24082892-69dc-465f-b406-756e8d658457 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:02.346369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 638/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:02.356290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.469524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc103c-33aa-4ab9-861f-a489a6a3a0b8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:02.507338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:02.508161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:02.508438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:02.508713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:02.519639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:02.521915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:47:02.539639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:02.540728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:02.545017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:47:02.545017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:47:02.545017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:47:02.545017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:02.545017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 640/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:02.548436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082892-69dc-465f-b406-756e8d658457 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.668563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24082892-69dc-465f-b406-756e8d658457 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:02.705417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:02.706084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:02.706368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:02.706638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:02.707462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:02.708295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:02.709677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:02.722568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 639/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:47:02.728655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.760904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:02.829254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025771d9-61c0-41b6-a1f0-437baa9ddacb None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:02.865287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:02.866090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:02.866373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:02.866645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] PATH_INFO: `/v3/users/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:02.874922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:02.876404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:47:02.898551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:02.900285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:02.901471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 641/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:47:02.906239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:02.912504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:03.014899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e366c-e5af-4a4f-b81b-15e7b09ffa2c None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:03.052211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:03.052867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:03.053147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:03.053422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:03.054283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:03.054563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:03.056030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:03.073385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 640/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Jul 14 10:47:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:47:03.079785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:03.193871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:03.204524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e886ff3-2cb4-4b59-8545-2cc6b58bb94b None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:03.240218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:03.240859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:03.241142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:03.241418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:03.313646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:03.315036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:03.408534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:03.419918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 642/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Jul 14 10:47:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:47:03.427011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:03.498070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:03.540651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5524498-7051-485d-98c4-733f24aeb00a None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:03.575687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:03.576509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:03.576782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:03.577051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:03.587257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:03.588740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:47:03.606928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:03.607861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:03.608832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:47:03.608832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:47:03.608832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:47:03.608832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:47:03.608832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 641/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:03.617271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:03.735697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabc264-8a9b-4d32-934e-73f5e88ce615 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:03.776770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:03.777425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:03.777698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:03.777971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:03.778681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:03.778985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:03.780397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:03.791684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 643/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Jul 14 10:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:03.800726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:03.905433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e716-050d-4045-8561-3062e9e6c281 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:03.922836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:03.946340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:03.947137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:03.947411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:03.947680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:03.954553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:03.956052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:47:03.969801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:03.970816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: ERROR keystone.server.flask.application Jul 14 10:47:03.976978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 642/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Jul 14 10:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:03.979274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:04.039224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:04.100672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28384a-b5d1-4311-8c91-f26e58b842dd None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:04.138073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:04.139003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:04.139365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:04.139950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:04.141134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:04.141562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:04.143819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81b28e-84bc-4a23-86b2-a409fe90abd5 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:04.162775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 644/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Jul 14 10:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:04.172058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:04.281624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04032b39-4ded-411c-9c2c-2d47b67f03e8 None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:04.317357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:04.317994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:04.318275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:04.318565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:04.319392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] RBAC: Adding query filter params (role.id=bd93d91744f54f739650facbdec5aef5, scope.project.id=b49330c5b3874cde8c864bc5b1a98a69, user.id=17f4edebe2ed45a08ef7460d812c563d) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:04.319705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:04.321432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af5037-0d6e-4c1e-8de1-78e6dd412736 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:04.327563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 643/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Jul 14 10:47:04 2022] GET /identity/v3/role_assignments?user.id=17f4edebe2ed45a08ef7460d812c563d&scope.project.id=b49330c5b3874cde8c864bc5b1a98a69&role.id=bd93d91744f54f739650facbdec5aef5 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:06.356730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289d8d9-d877-4b45-bc1a-1bd4abbb9881 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:06.357301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289d8d9-d877-4b45-bc1a-1bd4abbb9881 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:06.357301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289d8d9-d877-4b45-bc1a-1bd4abbb9881 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:06.358662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 645/1288] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:47:06.365247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:06.366001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:06.366140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:06.388754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:47:06.389959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:47:06.425906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:06.469113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 644/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:47:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:47:06.492682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289d8d9-d877-4b45-bc1a-1bd4abbb9881 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:06.613476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289d8d9-d877-4b45-bc1a-1bd4abbb9881 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:06.625004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:06.657608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:06.658452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:06.658801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:06.659127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:06.659819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:06.661520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:06.677338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:06.678417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:06.679301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 646/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:47:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:06.685144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:06.815858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf4002-d54c-4278-87d1-ec97892e7cce None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:06.865737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:06.866333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:06.866615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:06.866889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:06.867598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] RBAC: Adding query filter params (name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:06.867928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:06.869312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95c5bd-9ee4-4467-a55c-adf9a1db6132 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:06.874845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 645/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Jul 14 10:47:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:47:06.884754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:06.998039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6ff56-d41b-4059-9ee1-b5323fc2dbf8 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:07.034569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:07.035229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:07.035515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:07.036064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] PATH_INFO: `/v3/services` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:07.037065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] RBAC: Adding query filter params (type=placement) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:07.037380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:07.038769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44403c-313c-4e00-b543-e4f0bdba5e4c None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:07.043501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 647/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Jul 14 10:47:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:09.038929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8119-7dd7-4af1-be51-e11f9aedafe4 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:09.039772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8119-7dd7-4af1-be51-e11f9aedafe4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:09.039772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f8119-7dd7-4af1-be51-e11f9aedafe4 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:09.041943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 646/1293] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:47:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:47:09.050533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a20862-d04c-458b-859c-f5dd3658a52b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:09.050951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a20862-d04c-458b-859c-f5dd3658a52b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:09.051129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a20862-d04c-458b-859c-f5dd3658a52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:09.069291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d3a20862-d04c-458b-859c-f5dd3658a52b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:47:09.070471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d3a20862-d04c-458b-859c-f5dd3658a52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:47:09.100818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:09.140684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 648/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:47:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:47:09.171174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f8119-7dd7-4af1-be51-e11f9aedafe4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:09.210363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:09.296925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f8119-7dd7-4af1-be51-e11f9aedafe4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:09.332271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:09.332971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:09.333281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:09.333545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:09.334266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:09.335636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08826ab-132d-4de3-94ab-130d0843fc80 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:09.354035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 647/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Jul 14 10:47:09 2022] POST /identity/v3/services => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:47:11.372592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d7c6-43f8-4351-804f-e6846ebba641 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:11.373158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d7c6-43f8-4351-804f-e6846ebba641 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:11.373158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6d7c6-43f8-4351-804f-e6846ebba641 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:11.374519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 649/1296] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:47:11.381780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:11.382563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:11.382703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:11.407096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:47:11.408322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:47:11.493524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 648/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:47:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:47:11.522415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6d7c6-43f8-4351-804f-e6846ebba641 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:11.636411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6d7c6-43f8-4351-804f-e6846ebba641 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:11.672509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:11.673198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:11.673498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:11.673756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] PATH_INFO: `/v3/services/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:11.674430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:11.675804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:11.687858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:11.689385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:11.690524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 650/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:47:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:11.695752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:11.714346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:11.814859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f2efe-009b-4554-811f-a4592bf18e2c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:11.852871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:11.853541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:11.853820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:11.854091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:11.854797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] RBAC: Adding query filter params (name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:11.855099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:11.856529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:11.861880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 649/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Jul 14 10:47:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:11.869185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:11.903776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:11.977793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad8bbc-4d45-4fa9-852a-54f5c45cc993 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:12.014874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:12.015523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:12.015800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:12.016104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:12.016885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] RBAC: Adding query filter params (interface=public, service_id=2448f7eb76bf440292504ff3b0e61686, region_id=RegionOne) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:12.017191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:12.018567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66aa04c-62bf-4552-b241-528778a7bdfa None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:12.024281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 651/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Jul 14 10:47:11 2022] GET /identity/v3/endpoints?service_id=2448f7eb76bf440292504ff3b0e61686&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:12.032732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:12.119059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:12.147982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bdd38-96a9-4fd8-90f1-54dc1ec3d16b None admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:12.182976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:12.183541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:12.183817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:12.184116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:12.184788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:12.186159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9af6-3c4a-45a6-bf66-56f371283be7 None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:12.193492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 650/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Jul 14 10:47:12 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:47:14.201138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c51e5-a126-4b7d-a328-b435cf21afac None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:14.201949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c51e5-a126-4b7d-a328-b435cf21afac None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:14.201949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c51e5-a126-4b7d-a328-b435cf21afac None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:14.204228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 652/1302] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:47:14.211552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:14.212041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:14.212133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:14.230128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:47:14.231307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:47:14.301547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 651/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:47:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:47:14.337715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c51e5-a126-4b7d-a328-b435cf21afac None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:14.441771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:14.454446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c51e5-a126-4b7d-a328-b435cf21afac None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:14.489549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:14.490637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:14.490923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:14.491205 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] PATH_INFO: `/v3/services/placement` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:14.491859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:14.493287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:47:14.505890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:47:14.507011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application with Lock( Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: ERROR keystone.server.flask.application Jul 14 10:47:14.508570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 653/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Jul 14 10:47:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:47:14.517015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:14.604963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:14.628830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcca9f-e30e-4b04-bd07-632cb48f3367 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:14.663759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:14.664470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:14.664748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:14.665022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] PATH_INFO: `/v3/services` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:14.665730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:47:14.666036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:14.667420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad72e-9373-408b-9d57-f0585939d1cc None admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:14.672737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 652/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Jul 14 10:47:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:47:14.679519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:47:14.802097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a0375-4718-4849-b78b-88152556d179 None admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:47:14.839667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:47:14.840751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:47:14.841223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:47:14.841682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] PATH_INFO: `/v3/endpoints` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:47:14.842874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:47:14.845139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2478c5-5e99-44ca-a42e-fe4cdde98aed None admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:47:14.869011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 654/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Jul 14 10:47:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:47:24.669569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=30 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:47:24.863820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:18.749170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ac90-fd57-46f4-9d7f-33f1ca3f2962 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:18.749664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ac90-fd57-46f4-9d7f-33f1ca3f2962 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:18.749758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ac90-fd57-46f4-9d7f-33f1ca3f2962 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:18.751119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 653/1307] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:18.756752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae996ad-0d45-4995-9417-451843e26b4c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:18.757157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae996ad-0d45-4995-9417-451843e26b4c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:18.757433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae996ad-0d45-4995-9417-451843e26b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:18.903924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fae996ad-0d45-4995-9417-451843e26b4c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:18.906821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fae996ad-0d45-4995-9417-451843e26b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:19.052015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 655/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:19.100375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:19.101096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:19.101549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:19.104190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 654/1309] 10.222.0.27 () {54 vars in 964 bytes} [Thu Jul 14 10:48:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:19.109985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c682c-2b9f-40b7-93b5-296b8bf222ef None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:19.110345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c682c-2b9f-40b7-93b5-296b8bf222ef None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:19.110732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c682c-2b9f-40b7-93b5-296b8bf222ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:19.195273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-495c682c-2b9f-40b7-93b5-296b8bf222ef None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:19.197017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-495c682c-2b9f-40b7-93b5-296b8bf222ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:19.220374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:19.282689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 656/1310] 10.222.0.27 () {58 vars in 1061 bytes} [Thu Jul 14 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:19.291651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:19.293741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:19.414803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696d2e1-befa-40b3-987e-d242d1d4bfd7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:19.495336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:19.496169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:19.496518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:19.496855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:19.498595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:19.557796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:19.620345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:19.621950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5458a1-d835-4d11-9b0a-8aa4075640bc service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:19.645575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 655/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:21.987401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:21.987989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:21.987989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:21.989385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 657/1312] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:21.994786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552090c-5300-4d62-bb6b-7026c9589198 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:21.995209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552090c-5300-4d62-bb6b-7026c9589198 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:21.995346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552090c-5300-4d62-bb6b-7026c9589198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:22.086003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6552090c-5300-4d62-bb6b-7026c9589198 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:22.087541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6552090c-5300-4d62-bb6b-7026c9589198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:22.194450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 656/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:22.233879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:22.235394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:22.349583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e257f-366c-4dcd-8f09-a10fff94327a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:22.427798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:22.430675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:22.431367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:22.431652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:22.431928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:22.434178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:22.549159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:22.550635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe063605-130c-47f5-94ff-c87a21840c8f service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:22.570057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 658/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:24.901436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:24.901994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:24.901994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:24.903394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 657/1315] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:24.909721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecd599-decc-4b74-88d4-ec451576c0fe None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:24.910453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecd599-decc-4b74-88d4-ec451576c0fe None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:24.910590 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecd599-decc-4b74-88d4-ec451576c0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:24.978016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-edecd599-decc-4b74-88d4-ec451576c0fe None None] MFA Rules not processed for user `36056ea955f24614b41b93e8556945e6`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:24.979244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-edecd599-decc-4b74-88d4-ec451576c0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:25.035495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:25.081726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 659/1316] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:25.120926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:25.122485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:25.132523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:25.234279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f4ccb-3522-4583-8aa3-e2fe12437a10 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:25.310836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:25.311478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:25.311724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:25.312086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:25.313695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:25.376943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:25.428534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:25.430093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55fe81-5ae9-4672-b87a-90f900978b65 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:25.448366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 658/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:28.072689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:28.073244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:28.073244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:28.075384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 660/1318] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:28.084159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c8b4-3cc7-43fe-93a2-b1c31790e2a8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:28.084577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c8b4-3cc7-43fe-93a2-b1c31790e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:28.084760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32c8b4-3cc7-43fe-93a2-b1c31790e2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:28.157699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8b32c8b4-3cc7-43fe-93a2-b1c31790e2a8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:28.158890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8b32c8b4-3cc7-43fe-93a2-b1c31790e2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:28.271455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 659/1319] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:28.314362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:28.315906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:28.423022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5140fb-41a6-4c5e-af01-d7445ecfbe8e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:28.502591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:28.503242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:28.503518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:28.503783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:28.505454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:28.513210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:28.628089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:28.629611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c1b20-0057-4cda-a0fd-a5ac549c2962 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:28.648658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 661/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:48:31.384074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:31.384617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:31.384617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:31.386140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 660/1321] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:31.391554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c39d-8b9e-4ad8-8a3c-7f086d2b0ea0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:31.392048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c39d-8b9e-4ad8-8a3c-7f086d2b0ea0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:31.392143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3c39d-8b9e-4ad8-8a3c-7f086d2b0ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:31.459065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-78a3c39d-8b9e-4ad8-8a3c-7f086d2b0ea0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:31.460317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-78a3c39d-8b9e-4ad8-8a3c-7f086d2b0ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:31.522217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:31.568733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 662/1322] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:31.605279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:31.606904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:31.628545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:31.720398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a63b54-6d0a-486c-881a-f73dd55fed76 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:31.796385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:31.797039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:31.797323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:31.797596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:31.799250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:31.876228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:31.919538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:31.920974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f32dc-4c48-4ea8-bb39-f2366b1dcd07 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:31.943159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 661/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:34.005863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:34.006408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:34.006408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:34.007851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 663/1324] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:34.016997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316696d-c0fa-4445-8c20-197d740a84c2 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:34.018655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316696d-c0fa-4445-8c20-197d740a84c2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:34.018655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316696d-c0fa-4445-8c20-197d740a84c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:34.092631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f316696d-c0fa-4445-8c20-197d740a84c2 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:34.093872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f316696d-c0fa-4445-8c20-197d740a84c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:34.191393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 662/1325] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:34.226725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:34.228291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:34.337314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d488d9d-ed02-4865-8877-4a5912a92bc9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:34.416718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:34.417380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:34.417627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:34.417920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:34.419565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:34.437159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:34.530898 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:34.532504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae596d-a3e1-4ec8-959f-16da0ecb0c88 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:34.561545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 664/1326] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:36.674426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:36.674909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:36.675008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:36.676471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 663/1327] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:36.682201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3e4b-2259-471a-83dd-20c7353fafe1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:36.682660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3e4b-2259-471a-83dd-20c7353fafe1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:36.682762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf3e4b-2259-471a-83dd-20c7353fafe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:36.753071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-91cf3e4b-2259-471a-83dd-20c7353fafe1 None None] MFA Rules not processed for user `36056ea955f24614b41b93e8556945e6`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:36.754309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-91cf3e4b-2259-471a-83dd-20c7353fafe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:36.827345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:36.860817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 665/1328] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:36.899231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:36.900938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:36.933652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:37.014810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048184c-239f-44f3-ad1d-fe3e1ebc5fb8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:37.094189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:37.094852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:37.095117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:37.095444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:37.097214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:37.194356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:37.229831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:37.232029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8202f-6334-44f6-a96b-109b43610adf service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:37.251597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 664/1329] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:36 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:39.543371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065394a0-3e64-4d48-b323-dc5b59fcadfb None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:39.544011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065394a0-3e64-4d48-b323-dc5b59fcadfb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:39.544011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065394a0-3e64-4d48-b323-dc5b59fcadfb None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:39.545370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 666/1330] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:39.551749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399a5f-4179-49ba-ac83-2c327828d955 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:39.552555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399a5f-4179-49ba-ac83-2c327828d955 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:39.552733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399a5f-4179-49ba-ac83-2c327828d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:39.621987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9e399a5f-4179-49ba-ac83-2c327828d955 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:39.623196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9e399a5f-4179-49ba-ac83-2c327828d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:39.728423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 665/1331] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:39.779163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:39.779420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:39.779704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:39.781203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 667/1332] 10.222.0.27 () {54 vars in 964 bytes} [Thu Jul 14 10:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:39.787132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea37ca-e450-44f1-bce0-4126704475de None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:39.787441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea37ca-e450-44f1-bce0-4126704475de None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:39.787718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea37ca-e450-44f1-bce0-4126704475de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:39.832910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:39.868615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e1ea37ca-e450-44f1-bce0-4126704475de None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:39.869831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1ea37ca-e450-44f1-bce0-4126704475de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:39.946871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 666/1333] 10.222.0.27 () {58 vars in 1061 bytes} [Thu Jul 14 10:48:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:39.955859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:39.957405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:40.079696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989590-1eb0-4374-b6ed-25909c22614a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:40.167631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:40.168304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:40.168587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:40.168859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:40.170563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:40.200340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:40.299023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:40.300524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619fadf-0b35-401e-b9e5-c1cdb6d9ea6b service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:40.319135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 668/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:42.641116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:42.641702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:42.641702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:42.643097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 667/1335] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:42.649931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cc250-a605-4953-aeb5-a3bb0bac473b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:42.650378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cc250-a605-4953-aeb5-a3bb0bac473b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:42.650498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cc250-a605-4953-aeb5-a3bb0bac473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:42.726092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2b2cc250-a605-4953-aeb5-a3bb0bac473b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:42.727316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2b2cc250-a605-4953-aeb5-a3bb0bac473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:42.803417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:42.826741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 669/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:42.870770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:42.872387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:42.995448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8eff2-a543-40eb-9b7e-58677b49eaf1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:43.026485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:43.072908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:43.073475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:43.073753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:43.074028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:43.075684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:43.192176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:43.193712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e93d97-d312-4840-be16-b2eb1fa4ff15 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:43.213730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 668/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:45.911102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:45.911749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:45.911749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:45.913782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 670/1338] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:45.920301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121317fd-c91e-46af-b335-9938290fa52d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:45.920746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121317fd-c91e-46af-b335-9938290fa52d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:45.920884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121317fd-c91e-46af-b335-9938290fa52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:46.008500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-121317fd-c91e-46af-b335-9938290fa52d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:46.009706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-121317fd-c91e-46af-b335-9938290fa52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:46.018320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:46.123513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 669/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:46.163870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:46.165442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:46.283832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f9b50-2768-4127-a979-13ed34977099 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:46.359500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:46.360155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:46.360459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:46.360721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:46.362346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:46.399982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:46.494139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:46.495753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70858f4-4719-4f2e-9bef-57230a16deeb service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:46.515246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 671/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:50.705502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:50.706117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:50.706117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:50.707387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 670/1341] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:50.712954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74376d-c9c4-4310-8918-40af9ff3d2ff None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:50.713415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74376d-c9c4-4310-8918-40af9ff3d2ff None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:50.713539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74376d-c9c4-4310-8918-40af9ff3d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:50.778861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-da74376d-c9c4-4310-8918-40af9ff3d2ff None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:50.780099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-da74376d-c9c4-4310-8918-40af9ff3d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:50.867352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:50.878557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 672/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:50.918341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:50.919975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:51.026784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9228b-c080-4583-a787-2fbe3077a0f2 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:51.066246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:51.105722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:51.106382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:51.106664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:51.106939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:51.108623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:51.222729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:51.224405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33e30-0d46-4a82-b0bb-d918d21611cb service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:51.242997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 671/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:53.907634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:53.908258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:53.908258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:53.909630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 673/1344] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:53.916144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27175403-a245-441b-ab7a-1a36ee2397ed None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:53.916708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27175403-a245-441b-ab7a-1a36ee2397ed None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:53.916708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27175403-a245-441b-ab7a-1a36ee2397ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:53.988668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-27175403-a245-441b-ab7a-1a36ee2397ed None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:53.989934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-27175403-a245-441b-ab7a-1a36ee2397ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:54.010398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:54.095307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 672/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:54.133197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:54.134747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:54.242474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e2c01-8ad9-4b93-97e3-524ce1dd9791 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:54.323111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:54.323760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:54.324076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:54.324355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:54.326040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:54.369658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:54.438106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:54.439563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feab86-180a-410a-bfb5-22dceea50341 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:54.458926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 674/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:48:58.482913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:58.483442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:58.483442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:58.484798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 673/1347] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:48:58.491028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489a8-be17-4b01-a8b7-861f5cf2b3d3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:58.491494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489a8-be17-4b01-a8b7-861f5cf2b3d3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:58.491584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb489a8-be17-4b01-a8b7-861f5cf2b3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:58.561716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bcb489a8-be17-4b01-a8b7-861f5cf2b3d3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:48:58.563124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bcb489a8-be17-4b01-a8b7-861f5cf2b3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:58.673616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:58.677072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 675/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:48:58.715189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:48:58.716785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:58.831487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b346886-f75e-4dde-8b56-87d86f1ee1c1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:48:58.883426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:48:58.911457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:48:58.912140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:48:58.912421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:48:58.915685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:48:58.915685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:48:59.030169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:48:59.031629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9f39c-fe3f-4567-b6f2-c03e59306ad8 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:48:59.050265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 674/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:04.115780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef57d1-b7fb-43cb-8a5e-5d30e6e3b791 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:04.115780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef57d1-b7fb-43cb-8a5e-5d30e6e3b791 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:04.115780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef57d1-b7fb-43cb-8a5e-5d30e6e3b791 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:04.119830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 676/1350] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:49:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:04.127922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6c26-374b-413a-a5e2-19e40640979b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:04.127922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6c26-374b-413a-a5e2-19e40640979b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:04.127922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6c26-374b-413a-a5e2-19e40640979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:04.221288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-459c6c26-374b-413a-a5e2-19e40640979b None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:04.222391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-459c6c26-374b-413a-a5e2-19e40640979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:04.244890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:04.347085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 675/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:08.737965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc55d-7e83-4eea-bed4-0f6746385838 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:08.738581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc55d-7e83-4eea-bed4-0f6746385838 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:08.738829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc55d-7e83-4eea-bed4-0f6746385838 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:08.740550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 677/1352] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:08.745765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ce5d-f3d5-4b3a-8234-9a38cda32334 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:08.746206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ce5d-f3d5-4b3a-8234-9a38cda32334 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:08.746511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778ce5d-f3d5-4b3a-8234-9a38cda32334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:08.827932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d778ce5d-f3d5-4b3a-8234-9a38cda32334 None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:08.829327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d778ce5d-f3d5-4b3a-8234-9a38cda32334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:08.924579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:08.966982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 676/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:49:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:10.179423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d83a-0a50-480a-8279-e160408365b0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:10.180001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d83a-0a50-480a-8279-e160408365b0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:10.180001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d83a-0a50-480a-8279-e160408365b0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:10.181391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 678/1354] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:10.187493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1f440-7e24-419d-8c9c-d1ad80fe25a4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:10.187956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1f440-7e24-419d-8c9c-d1ad80fe25a4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:10.188219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1f440-7e24-419d-8c9c-d1ad80fe25a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:10.266074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fab1f440-7e24-419d-8c9c-d1ad80fe25a4 None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:10.267375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fab1f440-7e24-419d-8c9c-d1ad80fe25a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:10.376901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 677/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:13.864244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37614-27c7-481d-8255-43527a235580 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:13.864753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37614-27c7-481d-8255-43527a235580 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:13.864887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e37614-27c7-481d-8255-43527a235580 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:13.869199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 679/1356] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:13.873390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3af08-b285-444b-b66f-41a262e2b5aa None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:13.875200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3af08-b285-444b-b66f-41a262e2b5aa None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:13.875703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3af08-b285-444b-b66f-41a262e2b5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:13.928354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:13.965108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5dc3af08-b285-444b-b66f-41a262e2b5aa None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:13.966319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5dc3af08-b285-444b-b66f-41a262e2b5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:14.068130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 678/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:14.122029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359731-c59a-4352-a902-299b47d40d34 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:14.122634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359731-c59a-4352-a902-299b47d40d34 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:14.123254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46359731-c59a-4352-a902-299b47d40d34 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:14.125248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 680/1358] 10.222.0.27 () {54 vars in 961 bytes} [Thu Jul 14 10:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:14.135188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b76-96ee-479a-9a54-4dda322afdfb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:14.135801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b76-96ee-479a-9a54-4dda322afdfb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:14.137222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b76-96ee-479a-9a54-4dda322afdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:14.223688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f6999b76-96ee-479a-9a54-4dda322afdfb None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:14.224915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f6999b76-96ee-479a-9a54-4dda322afdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:14.277816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:14.349128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 679/1359] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Jul 14 10:49:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:49:14.356693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359731-c59a-4352-a902-299b47d40d34 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:14.358179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-46359731-c59a-4352-a902-299b47d40d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:14.522897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46359731-c59a-4352-a902-299b47d40d34 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:14.607812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:14.649658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:14.650305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:14.650586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:14.650860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:14.652497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:14.746874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:14.748676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6144b7c-784d-4c91-980c-3e627cee6f4d service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:14.767925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 681/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:17.002860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:17.003431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:17.003431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:17.005150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 680/1361] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:17.010753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49c728-c7c9-4e88-9bd9-384ea090d4fc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:17.011257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49c728-c7c9-4e88-9bd9-384ea090d4fc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:17.011257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49c728-c7c9-4e88-9bd9-384ea090d4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:17.080552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bc49c728-c7c9-4e88-9bd9-384ea090d4fc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:17.081771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bc49c728-c7c9-4e88-9bd9-384ea090d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:17.121158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:17.188603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 682/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:17.233676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:17.235243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:17.431342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88e0e4-250c-4df8-99eb-f3c8e4704153 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:17.443232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:17.554731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:17.555393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:17.555678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:17.555995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:17.557643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:17.662721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:17.664371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0683b-9b68-4c40-8d5d-25c17020b7cb service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:17.685110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 681/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:20.124958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2933-bc3a-4b09-afa6-35233a18ddca None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:20.125485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2933-bc3a-4b09-afa6-35233a18ddca None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:20.125485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f2933-bc3a-4b09-afa6-35233a18ddca None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:20.127218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 683/1364] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:20.132873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db595-eed0-4d13-b592-6a0f49a364bb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:20.133649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db595-eed0-4d13-b592-6a0f49a364bb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:20.134067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7db595-eed0-4d13-b592-6a0f49a364bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:20.154296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:20.214266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6a7db595-eed0-4d13-b592-6a0f49a364bb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:20.215378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6a7db595-eed0-4d13-b592-6a0f49a364bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:20.322876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 682/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:20.369826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:20.370197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:20.370544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:20.372333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 684/1366] 10.222.0.27 () {54 vars in 961 bytes} [Thu Jul 14 10:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:20.377488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f04d4-cda6-4101-aeb8-4fa9296fc1c3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:20.377741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f04d4-cda6-4101-aeb8-4fa9296fc1c3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:20.378043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f04d4-cda6-4101-aeb8-4fa9296fc1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:20.468847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-298f04d4-cda6-4101-aeb8-4fa9296fc1c3 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:20.469978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-298f04d4-cda6-4101-aeb8-4fa9296fc1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:20.507923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:20.615358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 683/1367] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Jul 14 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:20.624315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:20.625828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:20.778938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c979ab-acf2-4cbd-9f73-6955b5a68946 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:20.800539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:20.901631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:20.902310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:20.902589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:20.902862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:20.904533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:21.004895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:21.006521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3f647-4062-49a9-9b6e-ada1d1ad5c9f service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:21.028525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 685/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:21.976296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb595d-f9eb-44e6-af62-09a2c09b177b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:21.976296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb595d-f9eb-44e6-af62-09a2c09b177b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:21.976296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb595d-f9eb-44e6-af62-09a2c09b177b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:21.977927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 684/1369] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:21.983188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905b46f-adcd-4dae-8bfe-f88d1ddbba34 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:21.983701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905b46f-adcd-4dae-8bfe-f88d1ddbba34 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:21.983701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905b46f-adcd-4dae-8bfe-f88d1ddbba34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:22.014349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:22.050363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1905b46f-adcd-4dae-8bfe-f88d1ddbba34 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:22.051673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1905b46f-adcd-4dae-8bfe-f88d1ddbba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:22.151581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 686/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:22.497716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:22.498076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:22.498474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:22.500016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 685/1371] 10.222.0.27 () {54 vars in 963 bytes} [Thu Jul 14 10:49:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:22.505327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a075-389a-4d6e-ba75-c620d17cbf05 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:22.506517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a075-389a-4d6e-ba75-c620d17cbf05 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:22.506517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7a075-389a-4d6e-ba75-c620d17cbf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:22.584820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-07a7a075-389a-4d6e-ba75-c620d17cbf05 None None] MFA Rules not processed for user `13be832163b9406583f63f7cd00651ae`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:22.586064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-07a7a075-389a-4d6e-ba75-c620d17cbf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:22.640768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:22.674026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 687/1372] 10.222.0.27 () {58 vars in 1060 bytes} [Thu Jul 14 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:22.681898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:22.683433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:22.794990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65042-b1a6-45ec-b102-81f83a81725d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:22.836934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:22.874240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:22.874895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:22.875181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:22.875458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:22.877129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:23.008187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:23.009935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e32e64-d754-46c8-9364-4f934642314b service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:23.044210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 686/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:23.172062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:23.172513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:23.172740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:23.175149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 688/1374] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:23.180548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bfd-0cd4-4be8-b068-ef9693bc2c0b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:23.180548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bfd-0cd4-4be8-b068-ef9693bc2c0b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:23.181171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db61bfd-0cd4-4be8-b068-ef9693bc2c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:23.293516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0db61bfd-0cd4-4be8-b068-ef9693bc2c0b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:23.294759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0db61bfd-0cd4-4be8-b068-ef9693bc2c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:23.319068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:23.405513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 687/1375] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:23.448554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:23.450077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:23.604727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a44402-7c5d-48ef-94d1-6d0d80e8f51d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:23.658864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:23.725240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:23.725878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:23.726160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:23.726448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:23.728137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:23.822676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:23.824373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572db08-643b-4890-ad96-1f7b6bb1beb1 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:23.843144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 689/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:25.888593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:25.889111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:25.889209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:25.890655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 688/1377] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:25.896566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977e8bd-0bbf-418b-85b7-b7153bbc6e66 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:25.896973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977e8bd-0bbf-418b-85b7-b7153bbc6e66 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:25.897152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977e8bd-0bbf-418b-85b7-b7153bbc6e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:25.982875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c977e8bd-0bbf-418b-85b7-b7153bbc6e66 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:25.984281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c977e8bd-0bbf-418b-85b7-b7153bbc6e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:25.991455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:26.097953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 690/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:26.142787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:26.144730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:26.297180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab35cc-af70-4213-be92-ca0f55f55317 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:26.300941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:26.426051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:26.426767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:26.427046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:26.427315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:26.428965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:26.546313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:26.547947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd87596-e3cf-48e7-93be-e4f114724f1e service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:26.573882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 689/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:28.633792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:28.634330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:28.634330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:28.635865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 691/1380] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:28.641212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de31fa-18f3-4e86-8671-d0e4734f1841 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:28.642039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de31fa-18f3-4e86-8671-d0e4734f1841 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:28.642128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de31fa-18f3-4e86-8671-d0e4734f1841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:28.652647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:28.716206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-93de31fa-18f3-4e86-8671-d0e4734f1841 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:28.717721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-93de31fa-18f3-4e86-8671-d0e4734f1841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:28.827921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 690/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:49:28.869858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:28.871479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:29.024568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:29.036851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47370056-3272-4794-833a-e1c13cb7e23e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:29.155347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:29.156212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:29.156580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:29.156950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:29.158698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:29.252023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:29.254004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607546-1618-4e0a-85e4-5b749790caa4 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:29.265701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:29.276392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 692/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:31.346279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:31.346872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:31.346872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:31.348434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 691/1383] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:49:31.354090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a956-9449-4701-9e9e-e267fcaca701 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:31.354562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a956-9449-4701-9e9e-e267fcaca701 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:31.354777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a956-9449-4701-9e9e-e267fcaca701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:31.447107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0f60a956-9449-4701-9e9e-e267fcaca701 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:31.448398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0f60a956-9449-4701-9e9e-e267fcaca701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:31.560620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 693/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:31.607702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:31.609281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:31.725026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:31.794458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbc29c-a314-4290-8144-4c0d87dc7841 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:31.925893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:31.926456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:31.926748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:31.927036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:31.928815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:31.986864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:32.031898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:32.033538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e2f6d-43a5-4064-a411-deae90fcaca0 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:32.055575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 692/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:34.083702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:34.084277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:34.084277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:34.085681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 694/1386] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:34.091145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c690-c793-4823-96ca-4893785f4727 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:34.091594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c690-c793-4823-96ca-4893785f4727 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:34.091703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1c690-c793-4823-96ca-4893785f4727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:34.157867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-94d1c690-c793-4823-96ca-4893785f4727 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:34.159130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-94d1c690-c793-4823-96ca-4893785f4727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:34.263557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 693/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:49:34.308309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:34.309824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:34.388329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:34.470267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b8ae-e75e-4274-8258-f7e147d1c89d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:34.616518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:34.617781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:34.618727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:34.619209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:34.622622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:34.695744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:34.753771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:34.755464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485de59-75d9-486d-a52b-4099720b5f6c service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:34.776354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 695/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:36.941972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:36.942505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:36.942505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:36.943926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 694/1389] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:36.949601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9eb3-b9c9-4b19-be97-6590f365c4fb None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:36.950073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9eb3-b9c9-4b19-be97-6590f365c4fb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:36.950183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab9eb3-b9c9-4b19-be97-6590f365c4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:37.019524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-83ab9eb3-b9c9-4b19-be97-6590f365c4fb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:37.021077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-83ab9eb3-b9c9-4b19-be97-6590f365c4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:37.120119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 696/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:37.167280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:37.168920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:37.202914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:37.321193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c11fd5-9eb8-40eb-b6ea-59f3f2e4e6a9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:37.422691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:37.434819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:37.435500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:37.435777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:37.436070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:37.437703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:37.534616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:37.536171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cba7e-8301-4a74-86aa-54dcfa7a642f service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:37.555545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 695/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:38.250335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74907f6-ac16-4432-a315-776e963b6f28 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:38.250871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74907f6-ac16-4432-a315-776e963b6f28 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:38.250871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74907f6-ac16-4432-a315-776e963b6f28 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:38.252233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 697/1392] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:49:38.258054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8128c-bb5a-4fb4-bd2e-0f1bac04174d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:38.258534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8128c-bb5a-4fb4-bd2e-0f1bac04174d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:38.258655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8128c-bb5a-4fb4-bd2e-0f1bac04174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:38.330231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-eba8128c-bb5a-4fb4-bd2e-0f1bac04174d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:38.331519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-eba8128c-bb5a-4fb4-bd2e-0f1bac04174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:38.394648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:38.436580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 696/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:38.493559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:38.493856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:38.494153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:38.495544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 698/1394] 10.222.0.27 () {54 vars in 962 bytes} [Thu Jul 14 10:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:38.500768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252558-1114-4e31-aafd-f4bcfd976df2 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:38.501009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252558-1114-4e31-aafd-f4bcfd976df2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:38.501267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33252558-1114-4e31-aafd-f4bcfd976df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:38.579709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-33252558-1114-4e31-aafd-f4bcfd976df2 None None] MFA Rules not processed for user `44cc172ac2904c84a84c7f4dbb707901`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:38.581090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-33252558-1114-4e31-aafd-f4bcfd976df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:38.656064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 697/1395] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Jul 14 10:49:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:38.663691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:38.665331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:38.682253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:38.779294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2c6cc-7a25-455e-89ee-da7feceacf15 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:38.855823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:38.856501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:38.856780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:38.857049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:38.858673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:38.930859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:38.978830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:38.980335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f1337-8e2b-48ed-857e-878a91c3e273 service glance] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:38.998565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 699/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:39.615851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f259-0196-46f0-b650-e2a6a7c597fd None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:39.616495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f259-0196-46f0-b650-e2a6a7c597fd None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:39.616495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f259-0196-46f0-b650-e2a6a7c597fd None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:39.618556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 698/1397] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:39.623605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:39.624178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:39.624178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:39.690986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:39.692232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:39.714597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ea91-b0b4-4134-9b61-c04b6ba2d2d5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:39.714821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ea91-b0b4-4134-9b61-c04b6ba2d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:39.715191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ea91-b0b4-4134-9b61-c04b6ba2d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:39.778888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0ff1ea91-b0b4-4134-9b61-c04b6ba2d2d5 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:39.780117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0ff1ea91-b0b4-4134-9b61-c04b6ba2d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:39.788430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:39.796902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 700/1398] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:39.841669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:39.842883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:39.869140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:39.871365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 699/1399] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:49:39.922044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b602e-9b56-40bc-a053-e8447a46cd98 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:39.922372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b602e-9b56-40bc-a053-e8447a46cd98 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:39.922685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b602e-9b56-40bc-a053-e8447a46cd98 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:39.924112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 700/1400] 10.222.0.27 () {54 vars in 942 bytes} [Thu Jul 14 10:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:49:39.930461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:39.930728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:39.930998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:39.964252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee357178-26d6-4bb3-a8bc-9dd9a4bc4bb0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:40.007704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] MFA Rules not processed for user `6f958e0eb8444871851792cb254b56a7`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:40.009117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:40.079759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:40.080453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:40.080781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:40.081057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:40.082736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:40.094645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:40.101364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:40.118390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 701/1401] 10.222.0.27 () {58 vars in 1039 bytes} [Thu Jul 14 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:49:40.128321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:40.138081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdd46d-d92a-47dc-adc9-d93611bb0aaa None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:40.175259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:40.176942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cdb76-f712-464a-a68a-f55934f69575 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:40.195745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 701/1402] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:40.242555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:40.243212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:40.243489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:40.243767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:40.245515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:40.343260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:40.344972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05faf13b-f75a-4a18-9d34-116ddc5fc035 service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:40.355452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 702/1403] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:49:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 10:49:42.382102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:42.382725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:42.382725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:42.388108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 702/1404] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:42.390102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab194041-77fb-4cdd-87c9-9fe4d470a981 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:42.391360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab194041-77fb-4cdd-87c9-9fe4d470a981 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:42.391881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab194041-77fb-4cdd-87c9-9fe4d470a981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:42.402954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:42.465211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ab194041-77fb-4cdd-87c9-9fe4d470a981 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:42.466547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ab194041-77fb-4cdd-87c9-9fe4d470a981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:42.596234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 703/1405] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:42.631992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:42.633656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:42.809905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:42.816080 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32801c78-b87c-45af-9edf-d83aef7d0793 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:42.950520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:42.951511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:42.951858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:42.952177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:42.953856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:43.057733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:43.059730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf72a32-e2b4-41d0-92ac-bf12ccb77a23 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:43.079538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 703/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:45.154656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:45.155237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:45.155237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:45.156663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 704/1407] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:49:45.162112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b609b-9096-4ad0-877e-6f89d43e05a9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:45.162551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b609b-9096-4ad0-877e-6f89d43e05a9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:45.162654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b609b-9096-4ad0-877e-6f89d43e05a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:45.165970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:45.236348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2d3b609b-9096-4ad0-877e-6f89d43e05a9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:49:45.237534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2d3b609b-9096-4ad0-877e-6f89d43e05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:45.346072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 704/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:49:45.391256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:49:45.392918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:45.495643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:45.564104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c42711-39a9-4521-a9ca-692875927fc9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:49:45.721652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:49:45.723036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:49:45.723540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:49:45.724081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:49:45.726655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:49:45.838895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:45.890362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:49:45.891416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a5f93-6c69-4370-ba96-36bbe67cd2a2 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:49:45.916793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 705/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:49:55.344491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=64 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:49:55.914254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=23 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:02.291586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b349bdc-91e0-4a48-a6d3-a9fdd02184eb None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:02.292185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b349bdc-91e0-4a48-a6d3-a9fdd02184eb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:02.292185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b349bdc-91e0-4a48-a6d3-a9fdd02184eb None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:02.293497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 705/1410] 10.222.0.27 () {54 vars in 918 bytes} [Thu Jul 14 10:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:02.298995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c9ea-29eb-4211-a33b-89f0d16469ca None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:02.299459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c9ea-29eb-4211-a33b-89f0d16469ca None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:02.299553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c9ea-29eb-4211-a33b-89f0d16469ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:02.377945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3c79c9ea-29eb-4211-a33b-89f0d16469ca None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:02.379170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3c79c9ea-29eb-4211-a33b-89f0d16469ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:02.498677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 706/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Jul 14 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:02.513713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae000-5068-4f93-a0c1-f6db9360c4db None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:02.513996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae000-5068-4f93-a0c1-f6db9360c4db None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:02.514271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae000-5068-4f93-a0c1-f6db9360c4db None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:02.516085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 706/1412] 10.222.0.27 () {54 vars in 918 bytes} [Thu Jul 14 10:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:02.520843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e887-cdf8-4533-8cdd-a3ba239882e1 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:02.521149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e887-cdf8-4533-8cdd-a3ba239882e1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:02.521465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e887-cdf8-4533-8cdd-a3ba239882e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:02.597282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:02.600791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5b50e887-cdf8-4533-8cdd-a3ba239882e1 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:02.602823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5b50e887-cdf8-4533-8cdd-a3ba239882e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:02.730238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 707/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Jul 14 10:50:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:05.242957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:05.243782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:05.243782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:05.245912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:05.252414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e572-9ad0-4c30-9af2-b10d9dce034f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:05.253156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e572-9ad0-4c30-9af2-b10d9dce034f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:05.253315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e572-9ad0-4c30-9af2-b10d9dce034f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:05.325130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8c05e572-9ad0-4c30-9af2-b10d9dce034f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:05.326332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8c05e572-9ad0-4c30-9af2-b10d9dce034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:05.387498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:05.431796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:05.474531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:05.476160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:05.639439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3904-9d77-438c-b03e-a0eda238443c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:05.725104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:05.771795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:05.772473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:05.772745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:05.773011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:05.774734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:05.867893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:05.869512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483e5af-400d-463d-9de8-0c638236a8b0 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:05.887929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 708/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:06.783076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:06.784029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:06.784029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:06.786242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:06.791385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e2a35-fdbc-413c-af3c-ffed52dd8cb9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:06.791841 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e2a35-fdbc-413c-af3c-ffed52dd8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:06.791968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e2a35-fdbc-413c-af3c-ffed52dd8cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:06.869104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d52e2a35-fdbc-413c-af3c-ffed52dd8cb9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:06.870387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d52e2a35-fdbc-413c-af3c-ffed52dd8cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:06.898996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:06.975739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:07.130417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:07.134705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:07.302265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8dd3-677b-47b8-831b-73c8d72fdf3b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:07.344732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:07.422663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:07.423242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:07.423841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:07.423841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:07.425451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:07.523604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:07.525256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ff4d5-1d66-4287-8e8f-b33e800e8ade service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:07.543677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 710/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:08.455282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:08.455862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:08.455862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:08.457245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:08.465588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca73b-58c2-4994-b157-348a7c87028f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:08.466278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca73b-58c2-4994-b157-348a7c87028f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:08.466513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaca73b-58c2-4994-b157-348a7c87028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:08.524456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:08.547572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-deaca73b-58c2-4994-b157-348a7c87028f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:08.549061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-deaca73b-58c2-4994-b157-348a7c87028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:08.653641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:08.808597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:08.811441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:08.972245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fb57f-5e51-48f4-9b3c-b1032d39bc16 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:08.984515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:09.093427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:09.093993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:09.094279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:09.094554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:09.096333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:09.192279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:09.193897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c58853-6bf4-4392-a5c2-6a1b76c4184e service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:09.213558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 711/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:10.084890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:10.085485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:10.085485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:10.086850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:10.092976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b79d6-fe0b-48c9-8f20-ccf6b223b6e3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:10.093734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b79d6-fe0b-48c9-8f20-ccf6b223b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:10.093881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b79d6-fe0b-48c9-8f20-ccf6b223b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:10.114349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:10.175789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-660b79d6-fe0b-48c9-8f20-ccf6b223b6e3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:10.176966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-660b79d6-fe0b-48c9-8f20-ccf6b223b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:10.277715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:10.435961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:10.438195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:10.588535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:10.625642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5a23-a853-45fe-924b-9f7de157b219 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:10.745034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:10.745599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:10.745879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:10.746147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:10.747801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:10.840726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:10.855839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:10.857482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafbcb8-b91f-4224-8dc8-caddb0c316b2 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:10.875597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 713/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:11.747161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:11.748095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:11.748095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:11.750403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:11.756594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acae5c-807a-46e3-b96e-3ec63ebe0763 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:11.757382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acae5c-807a-46e3-b96e-3ec63ebe0763 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:11.757536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acae5c-807a-46e3-b96e-3ec63ebe0763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:11.838048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c6acae5c-807a-46e3-b96e-3ec63ebe0763 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:11.839244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c6acae5c-807a-46e3-b96e-3ec63ebe0763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:11.940860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:12.094462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:12.096028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:12.190375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:12.263787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03a6bf-e761-45bc-ab79-50dc13f24973 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:12.383445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:12.384107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:12.384383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:12.384652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:12.386292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:12.437668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:12.482563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:12.484221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f45d20-c86a-4806-82ac-739bc6d17cc2 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:12.509779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 714/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:13.327527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:13.328489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:13.328489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:13.330294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:13.335619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3df847-3f92-4497-a102-5a53174d4091 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:13.336098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3df847-3f92-4497-a102-5a53174d4091 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:13.336201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3df847-3f92-4497-a102-5a53174d4091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:13.414348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8b3df847-3f92-4497-a102-5a53174d4091 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:13.415594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8b3df847-3f92-4497-a102-5a53174d4091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:13.522232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:13.674837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:13.677262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:13.733050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:13.836164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4080218-b24b-4ccd-975f-8a7d4b04a9f6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:13.963464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:13.964198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:13.964479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:13.964753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:13.966452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:13.974153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:14.072271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:14.073875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209babfe-aac2-454e-a3bb-3ec73a12f7ed service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:14.093484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 716/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:14.909451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:14.910041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:14.910041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:14.912196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:14.920442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7702-bddf-4d58-a11a-218625a4a443 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:14.921164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7702-bddf-4d58-a11a-218625a4a443 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:14.921328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae7702-bddf-4d58-a11a-218625a4a443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:14.998094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-30ae7702-bddf-4d58-a11a-218625a4a443 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:14.999306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-30ae7702-bddf-4d58-a11a-218625a4a443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:15.070570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:15.100283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:15.261759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:15.264134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:15.292500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:15.426033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6704b3-6017-4484-b69d-16b52f210d4e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:15.518162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:15.545335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:15.545986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:15.546266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:15.546536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:15.548289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:15.645718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:15.647335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e1efd-1275-48f1-be28-94b9fed6d9d3 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:15.665440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 717/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:16.490177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:16.491051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:16.491051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:16.493355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:16.500118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e06247-0949-46b0-9b2d-16810da12742 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:16.500717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e06247-0949-46b0-9b2d-16810da12742 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:16.500717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e06247-0949-46b0-9b2d-16810da12742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:16.582702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-92e06247-0949-46b0-9b2d-16810da12742 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:16.583904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-92e06247-0949-46b0-9b2d-16810da12742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:16.627988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:16.692738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:16.844920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:16.847200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:17.012296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd17b8c-95d2-4dd3-bd5c-c56268632e28 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:17.068492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:17.128162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:17.128814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:17.129091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:17.129367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:17.131013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:17.232947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:17.234556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d092b-c79e-48d2-96fe-add5cb5955ee service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:17.253612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 719/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:18.075037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:18.075571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:18.075571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:18.077053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:18.082314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fedad-414b-44d9-893e-414956e61848 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:18.082809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fedad-414b-44d9-893e-414956e61848 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:18.082905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fedad-414b-44d9-893e-414956e61848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:18.159470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3f0fedad-414b-44d9-893e-414956e61848 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:18.160737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3f0fedad-414b-44d9-893e-414956e61848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:18.162809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:18.262342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:50:18.413203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:18.415704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:18.576751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072419-20cf-458a-ab3e-2b951da31a81 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:18.604064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:18.696231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:18.696885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:18.697170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:18.697443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:18.699107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:18.799904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:18.801469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985227d0-1a2d-4e2b-bffa-ba8c236db9dd service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:18.821276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 720/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:20.799191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:20.800143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:20.800143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:20.802523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:20.808929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eaa74-62eb-4efc-a908-18fdaba31058 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:20.809388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eaa74-62eb-4efc-a908-18fdaba31058 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:20.809484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eaa74-62eb-4efc-a908-18fdaba31058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:20.855765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:20.882904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-732eaa74-62eb-4efc-a908-18fdaba31058 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:20.884221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-732eaa74-62eb-4efc-a908-18fdaba31058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:20.986308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:21.035739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:21.037312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:21.156516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875828d-664d-42a8-a7af-3df3e66175fa None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:21.191244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:21.245008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:21.245657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:21.245934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:21.246210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:21.247841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:21.369375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:21.370825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff06b5-b6fa-4a92-87a9-724c04881411 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:21.399999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 722/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:23.548147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:23.549057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:23.549057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:23.550413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:23.557873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a00c-e3af-45df-a34c-09b68dc15c6d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:23.558292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a00c-e3af-45df-a34c-09b68dc15c6d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:23.558475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a00c-e3af-45df-a34c-09b68dc15c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:23.770522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5921a00c-e3af-45df-a34c-09b68dc15c6d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:23.771739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5921a00c-e3af-45df-a34c-09b68dc15c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:23.785274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:23.885947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:23.925258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:23.927385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:24.047076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f26a7-313a-420f-bb2b-b7a4fff04169 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:24.060834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:24.124629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:24.125339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:24.125769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:24.126035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:24.127721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:24.242055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:24.243417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3a78-1ebb-49e7-bcde-75fbad79e317 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:24.262472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 723/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:26.646595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:26.647178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:26.647178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:26.648580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:26.655091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02272c-adff-4aab-8894-7c07e4e08cab None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:26.655808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02272c-adff-4aab-8894-7c07e4e08cab None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:26.656015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02272c-adff-4aab-8894-7c07e4e08cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:26.715672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:26.736374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9a02272c-adff-4aab-8894-7c07e4e08cab None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:26.737598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9a02272c-adff-4aab-8894-7c07e4e08cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:26.839986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:26.883866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:26.885625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:27.014656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df6517-d559-421e-92a4-2a8cc49b78e5 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:27.056753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:27.092702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:27.093274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:27.093544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:27.093833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:27.095987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:27.210159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:27.211577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfdd01-7a25-49a4-8c22-4a76de4da2aa service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:27.232459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 725/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:29.483995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:29.484844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:29.484844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:29.485970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:29.492221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ebb1-6f33-4f7c-a04c-347729cc6848 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:29.492686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ebb1-6f33-4f7c-a04c-347729cc6848 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:29.492801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ebb1-6f33-4f7c-a04c-347729cc6848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:29.566315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-af79ebb1-6f33-4f7c-a04c-347729cc6848 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:29.567514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-af79ebb1-6f33-4f7c-a04c-347729cc6848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:29.588486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:29.667449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:29.706883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:29.708501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:29.816002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17f254-7476-4bb9-a110-676d78dac6eb None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:29.839959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:29.901313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:29.901962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:29.902246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:29.902520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:29.904207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:30.022555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:30.024064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b32e5-5187-4248-9912-befd6a93ce2b service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:30.043760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 726/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:32.132596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:32.133452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:32.133452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:32.135574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:32.145203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070cc9a-bf1f-4b4c-9296-2c4dc236bd47 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:32.145657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070cc9a-bf1f-4b4c-9296-2c4dc236bd47 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:32.145755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070cc9a-bf1f-4b4c-9296-2c4dc236bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:32.231670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:32.235008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0070cc9a-bf1f-4b4c-9296-2c4dc236bd47 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:32.237386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0070cc9a-bf1f-4b4c-9296-2c4dc236bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:32.351568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:32.393823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:32.395354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:32.518235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84bab1-c788-457f-ab3a-7592d50bdab0 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:32.576093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:32.602164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:32.602829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:32.603106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:32.603374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:32.605019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:32.718082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:32.719702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ded89b-06a5-4209-b4c7-541a9a7f4614 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:32.738554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 728/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:34.891134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:34.892166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:34.892166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:34.894418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:34.900847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239793-4ddd-4742-9f45-971fc7687ac9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:34.901320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239793-4ddd-4742-9f45-971fc7687ac9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:34.901417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32239793-4ddd-4742-9f45-971fc7687ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:34.992814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-32239793-4ddd-4742-9f45-971fc7687ac9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:34.994228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-32239793-4ddd-4742-9f45-971fc7687ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:35.030610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:35.108461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:35.150913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:35.153397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:35.275011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfcbcc-639b-4e64-9273-868ddcb2f1b4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:35.305645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:35.355958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:35.356611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:35.356890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:35.357160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:35.358821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:35.495521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:35.497029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b75315-fcb8-4163-b90a-fe3bb89c610f service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:35.518050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 729/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:37.626729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:37.627300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:37.627397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:37.629694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:37.639156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d25eca-7131-49fc-a477-395f030fc185 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:37.639569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d25eca-7131-49fc-a477-395f030fc185 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:37.639772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d25eca-7131-49fc-a477-395f030fc185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:37.727247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-03d25eca-7131-49fc-a477-395f030fc185 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:37.728546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-03d25eca-7131-49fc-a477-395f030fc185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:37.737667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:37.854852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:37.898992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:37.900611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:38.016647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f66d6-f731-4c2f-adee-8082b322a240 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:38.080192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:38.099255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:38.099910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:38.100205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:38.100477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:38.102118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:38.233317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:38.235119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a0a28-9b89-4668-94bb-fa235a75483f service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:38.254664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 731/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:40.364134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:40.364688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:40.364688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:40.366813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:40.376210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d204e85-c99f-4857-989d-2c67d6888d7e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:40.376703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d204e85-c99f-4857-989d-2c67d6888d7e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:40.377413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d204e85-c99f-4857-989d-2c67d6888d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:40.455397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0d204e85-c99f-4857-989d-2c67d6888d7e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:40.456640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0d204e85-c99f-4857-989d-2c67d6888d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:40.497729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:40.567151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:40.606001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:40.608568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:40.741816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a6010-8b24-4225-9243-75c89b737f10 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:40.791571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:40.834541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:40.835181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:40.835481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:40.835749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:40.837539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:40.953907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:40.955372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db7cf9-5c82-4ce2-a003-a9ff2daf8470 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:40.974125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 732/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:43.066612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:43.067208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:43.067208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:43.068993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:43.075467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1c2c-eccc-487f-b0ea-408153604f01 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:43.076038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1c2c-eccc-487f-b0ea-408153604f01 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:43.076219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1c2c-eccc-487f-b0ea-408153604f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:43.151393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d6bd1c2c-eccc-487f-b0ea-408153604f01 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:43.153220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d6bd1c2c-eccc-487f-b0ea-408153604f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:43.173754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:43.254794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:43.294998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:43.296645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:43.425571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1e3c1-af86-4474-aad7-8a658b514eb1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:43.501716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:43.509959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:43.510565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:43.510847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:43.511118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:43.512851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:43.634140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:43.635599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4b8f8-e0eb-4137-bef4-2dc312b7b5ae service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:43.654011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 734/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:45.698039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:45.698925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:45.698925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:45.701266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:50:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:50:45.706924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7457b-b4c7-4b44-8061-35ec832c185d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:45.707387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7457b-b4c7-4b44-8061-35ec832c185d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:45.707502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7457b-b4c7-4b44-8061-35ec832c185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:45.776251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-43a7457b-b4c7-4b44-8061-35ec832c185d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:50:45.777486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-43a7457b-b4c7-4b44-8061-35ec832c185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:45.828180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:45.881227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:50:45.917876 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:50:45.919454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:46.029182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73345d90-4bf6-4c05-85cb-d7e2e931bb07 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:50:46.077300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:46.109290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:50:46.109921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:50:46.110201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:50:46.110469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:50:46.112148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:50:46.234497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:50:46.236005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346c75f-c2b0-4510-a140-4f917e4e2729 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:50:46.253992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 735/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:50:55.879215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=22 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:50:56.251792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=66 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:06.241129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3b8e0-34cb-4ea0-a92d-8ccb7dd5421c None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:06.241675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3b8e0-34cb-4ea0-a92d-8ccb7dd5421c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:06.241675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3b8e0-34cb-4ea0-a92d-8ccb7dd5421c None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:06.242997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 736/1471] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:06.249815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8794d5-f349-44a8-9784-17dc84aab053 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:06.250589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8794d5-f349-44a8-9784-17dc84aab053 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:06.250739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8794d5-f349-44a8-9784-17dc84aab053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:06.324927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4c8794d5-f349-44a8-9784-17dc84aab053 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:06.326130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4c8794d5-f349-44a8-9784-17dc84aab053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:06.429337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 736/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:06.517600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:06.517894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:06.518171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:06.519713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 737/1473] 10.222.0.27 () {54 vars in 962 bytes} [Thu Jul 14 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:06.526526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cf48-01e7-4a2e-b2c2-ae129731ea29 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:06.526884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cf48-01e7-4a2e-b2c2-ae129731ea29 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:06.527251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cf48-01e7-4a2e-b2c2-ae129731ea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:06.610616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1e29cf48-01e7-4a2e-b2c2-ae129731ea29 None None] MFA Rules not processed for user `44cc172ac2904c84a84c7f4dbb707901`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:06.611929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1e29cf48-01e7-4a2e-b2c2-ae129731ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:06.634894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:06.696094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 737/1474] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Jul 14 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:06.703411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:06.704992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:06.813308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c7dc2-813c-4428-b7d2-7e7b4a9e5580 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:06.890294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c5911-500b-44f5-835c-06807132356c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:06.890948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c5911-500b-44f5-835c-06807132356c service glance] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:06.891225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c5911-500b-44f5-835c-06807132356c service glance] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:06.891499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c5911-500b-44f5-835c-06807132356c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:06.893163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-549c5911-500b-44f5-835c-06807132356c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:06.962153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:07.019550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c5911-500b-44f5-835c-06807132356c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:07.021024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c5911-500b-44f5-835c-06807132356c service glance] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:07.045711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 738/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:51:09.165950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:09.166515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:09.166515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:09.168622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 738/1476] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:09.175448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce66fff-f67f-4c02-b320-90d98e50217c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:09.175862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce66fff-f67f-4c02-b320-90d98e50217c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:09.176031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce66fff-f67f-4c02-b320-90d98e50217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:09.263130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cce66fff-f67f-4c02-b320-90d98e50217c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:09.264362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cce66fff-f67f-4c02-b320-90d98e50217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:09.370590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 739/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:09.402063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:09.403637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:09.537942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd03c9e-6977-4cd0-b0ee-6ab10a40a362 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:09.597978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:09.643899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:09.644576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:09.644853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:09.645128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:09.645865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:09.647687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:51:09.647687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:51:09.647687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:51:09.648897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68190d-5e82-4536-bd5a-064bdfa20afa admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:09.667912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 739/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Jul 14 10:51:09 2022] GET /identity/v3/projects => generated 2553 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:51:11.743909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:11.744514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:11.744514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:11.745814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 740/1479] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:11.752404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323830-91b8-4328-80cc-092c4f8975fb None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:11.752826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323830-91b8-4328-80cc-092c4f8975fb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:11.752979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323830-91b8-4328-80cc-092c4f8975fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:11.839772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4a323830-91b8-4328-80cc-092c4f8975fb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:11.841742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4a323830-91b8-4328-80cc-092c4f8975fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:11.952985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 740/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:11.997071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:11.999158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:12.009080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:12.158912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fa72c-38f1-4d00-99ca-deddba384a85 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:12.256298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:12.294320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:12.295003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:12.295547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:12.295547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:12.297250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:12.400488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:12.402089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3818d-dbf4-4922-bbaa-6838de5a5d8e service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:12.420403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 741/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:51:14.632279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:14.632835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:14.632835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:14.634207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 741/1482] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:14.640084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176a64-e963-4ce6-9b69-94ddfcbd9148 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:14.640582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176a64-e963-4ce6-9b69-94ddfcbd9148 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:14.640582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c176a64-e963-4ce6-9b69-94ddfcbd9148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:14.706220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3c176a64-e963-4ce6-9b69-94ddfcbd9148 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:14.707413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3c176a64-e963-4ce6-9b69-94ddfcbd9148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:14.753365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:14.817020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 742/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:14.862012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:14.863560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:14.888441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:14.979724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fadbd-6a96-4c83-9c4d-e9d4703ddcd8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:15.056963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:15.057523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:15.057802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:15.058073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:15.059721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:15.146079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:15.193320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:15.194773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894fb16-2feb-4df3-a4ea-7ce5de5ed0ab service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:15.216379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 742/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:51:17.296835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:17.297363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:17.297363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:17.299342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 743/1485] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:17.306123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817e43-50b8-4630-a62e-f76b2d13adc8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:17.306559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817e43-50b8-4630-a62e-f76b2d13adc8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:17.306715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817e43-50b8-4630-a62e-f76b2d13adc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:17.392058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-78817e43-50b8-4630-a62e-f76b2d13adc8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:17.393282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-78817e43-50b8-4630-a62e-f76b2d13adc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:17.493967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 743/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:17.537060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:17.538778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:17.665088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb787b-a926-4157-83e4-b43a5b570b66 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:17.743773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:17.746142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:17.747203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:17.747778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:17.748298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:17.750631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:17.877287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:17.879360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cae335-665a-488a-90c5-177d3898b8a7 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:17.908072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 744/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:51:20.126966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:20.127564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:20.127564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:20.128945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 744/1488] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:20.135299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91c0d-c841-46e5-84f0-a0a9e6452632 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:20.135754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91c0d-c841-46e5-84f0-a0a9e6452632 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:20.135959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda91c0d-c841-46e5-84f0-a0a9e6452632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:20.213703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-bda91c0d-c841-46e5-84f0-a0a9e6452632 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:20.214816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bda91c0d-c841-46e5-84f0-a0a9e6452632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:20.272912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:20.320681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 745/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:20.362411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:20.363993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:20.403469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:20.477543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b250f9-e34b-4cb6-8fe1-9421a5fa635b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:20.563558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:20.564231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:20.564512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:20.564778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:20.566413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:20.652621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:20.687437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:20.688915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804b931-4b39-45f0-9be9-92026db05d01 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:20.708046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 745/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:51:22.998380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:22.998917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:22.998917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:23.000409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 746/1491] 10.222.0.27 () {54 vars in 926 bytes} [Thu Jul 14 10:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:51:23.006786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a57-eaa7-43f2-bd2a-b6ad5a584eb9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:23.007417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a57-eaa7-43f2-bd2a-b6ad5a584eb9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:23.007529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a57-eaa7-43f2-bd2a-b6ad5a584eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:23.081227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ec253a57-eaa7-43f2-bd2a-b6ad5a584eb9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:51:23.082451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ec253a57-eaa7-43f2-bd2a-b6ad5a584eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:23.190498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 746/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Thu Jul 14 10:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:51:23.236407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:51:23.237978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:23.353437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b31ea-43b3-478c-ab10-5dcaf294a462 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:51:23.443449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:51:23.444050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:51:23.444336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:51:23.444583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:51:23.446282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:51:23.456619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:23.566943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:51:23.568431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314133da-25e0-47d0-9c8c-fae9d8daa278 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:51:23.587361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 747/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:51:33.188574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=82 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:51:33.585118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=50 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:49.987962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:49.988552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:49.988552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.068015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:50.069217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.168965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 747/1494] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:52:50.177506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.178014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.178014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.283512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:50.284752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.393773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 748/1495] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:52:50.399707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:50.400971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.412814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff92520-176c-42b7-8ee5-1d4d769d7d35 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:50.477521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:50.515364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:50.516049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.516327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.516595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.517319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:52:50.517621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:50.518649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:50.518649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:52:50.518649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:52:50.519561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:50.530695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 748/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:52:50.539099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:50.548603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691e45-caba-4d9b-96e1-15f36b04792f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:50.622750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:50.657474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:50.658145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.658422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.658691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.659429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:50.661406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:50.661406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:52:50.661406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:52:50.662637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:50.724935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 749/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:52:50 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:50.729530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:50.731373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.766764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0e28f-7a55-45b2-86d9-2d287231761d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:50.866790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:50.867463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:50.867744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:50.868050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:50.868741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:50.869771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:50.869771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:52:50.869771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:52:50.870968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:50.914557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 749/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:52:50 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:52:50.918812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:50.920507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:50.985980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad66760-74aa-4a14-9bee-a6c94bc99af5 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.080814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.092514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.093192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.093473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.093750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.094463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.095490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:51.095490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:52:51.095490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:52:51.096409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.108993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 750/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:52:51.113411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.114947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:51.128807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a603-92a1-4b36-bc15-b890dfa39879 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.158814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.234859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.235534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.235816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.236183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.236860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.237933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:51.237933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:52:51.237933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:52:51.238827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082d0f-33a5-4758-bbf7-4373a516180d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.251995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 750/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:52:51.256613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.258172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:51.300353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df86d8-ecae-4237-a54c-6b1c6753c7c6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:51.407942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:51.408861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.409137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.409407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] PATH_INFO: `/v3/projects/032e48429dcb45b682bcef79f0ea27f5/users/b74ad87dbbd243f6a0313cfc1c7747e8/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.439136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032e48429dcb45b682bcef79f0ea27f5, user_id=b74ad87dbbd243f6a0313cfc1c7747e8, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:51.440617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:51.440617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:52:51.440617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:52:51.443523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da7c2c-1395-4c32-a50f-25e203722cd1 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:51.476642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 751/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:52:51 2022] PUT /identity/v3/projects/032e48429dcb45b682bcef79f0ea27f5/users/b74ad87dbbd243f6a0313cfc1c7747e8/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:52:51.489282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a1c2-ae9e-4481-9645-8c16837d537e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.489464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a1c2-ae9e-4481-9645-8c16837d537e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.489755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a1c2-ae9e-4481-9645-8c16837d537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.548419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fbe7a1c2-ae9e-4481-9645-8c16837d537e None None] MFA Rules not processed for user `b74ad87dbbd243f6a0313cfc1c7747e8`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:51.549655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fbe7a1c2-ae9e-4481-9645-8c16837d537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:51.630226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 751/1502] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:52:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:52:51.644895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.645183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.645480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.646906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 752/1503] 10.222.0.27 () {54 vars in 963 bytes} [Thu Jul 14 10:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:51.654022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dad4c-0828-433a-bd51-0343033055c7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:51.654314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dad4c-0828-433a-bd51-0343033055c7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:51.654589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dad4c-0828-433a-bd51-0343033055c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:51.686655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.734418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d17dad4c-0828-433a-bd51-0343033055c7 None None] MFA Rules not processed for user `13be832163b9406583f63f7cd00651ae`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:51.735629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d17dad4c-0828-433a-bd51-0343033055c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:51.827515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 752/1504] 10.222.0.27 () {58 vars in 1060 bytes} [Thu Jul 14 10:52:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:52:51.835605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:51.837171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:51.890811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:51.945064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b28fb-a775-42bc-9774-7bb86942c852 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:52.020014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:52.020696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:52.020981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:52.021254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:52.022902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:52.135687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:52.137592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:52.141198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:52.157303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 753/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:52:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:52:52.208142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198a8b-704b-48ae-b9cd-3ce417c07d37 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:52.208622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198a8b-704b-48ae-b9cd-3ce417c07d37 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:52.208724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41198a8b-704b-48ae-b9cd-3ce417c07d37 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:52.209984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 753/1506] 10.222.0.27 () {52 vars in 829 bytes} [Thu Jul 14 10:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:52:52.213894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:52.215597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:52.321481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83840c95-7de3-4520-9e94-aa5ec5d186a1 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:52.423561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:52.424493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:52.424835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:52.425168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] PATH_INFO: `/v3/users/b74ad87dbbd243f6a0313cfc1c7747e8` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:52.439087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74ad87dbbd243f6a0313cfc1c7747e8)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:52.440258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:52.440258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:52:52.440258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:52:52.441496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:52.451973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:52.508405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74ad87dbbd243f6a0313cfc1c7747e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:52.529826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74ad87dbbd243f6a0313cfc1c7747e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:52.530409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74ad87dbbd243f6a0313cfc1c7747e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:52.543234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74ad87dbbd243f6a0313cfc1c7747e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:52.555742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74ad87dbbd243f6a0313cfc1c7747e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:52.561674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 754/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:52:52 2022] DELETE /identity/v3/users/b74ad87dbbd243f6a0313cfc1c7747e8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:52:52.566192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:52.566482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:52.566756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:52.621724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:52:52.622915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:52.697595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:52.711487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 754/1508] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:52:52.722388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:52.723657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:52.838442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef5b-5b95-4d08-a193-865fbec7905a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:52.909773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:52:52.931724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:52.932415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:52.932716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:52.932977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:52.934867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:53.050070 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:53.051400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b7f99-963c-44b4-b7ee-6a5b02b68c39 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:53.074806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 755/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:52:53.451012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:52:53.452547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:52:53.466129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917029-02f2-463b-849f-d6d2ea672b59 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:52:53.565941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:52:53.566745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:52:53.567028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:52:53.567304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] PATH_INFO: `/v3/projects/032e48429dcb45b682bcef79f0ea27f5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:52:53.579483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032e48429dcb45b682bcef79f0ea27f5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:52:53.580602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:52:53.580602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:52:53.580602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:52:53.581863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:52:53.648499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032e48429dcb45b682bcef79f0ea27f5', 'request_id': 'req-edf806e3-b067-4505-badb-3472a03b0872'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:53.663467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032e48429dcb45b682bcef79f0ea27f5', 'request_id': 'req-edf806e3-b067-4505-badb-3472a03b0872'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:53.669916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032e48429dcb45b682bcef79f0ea27f5', 'request_id': 'req-edf806e3-b067-4505-badb-3472a03b0872'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:53.700957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] The token cache is being invalidate because project 032e48429dcb45b682bcef79f0ea27f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:52:53.701244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:53.701760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-edf806e3-b067-4505-badb-3472a03b0872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:52:53.703120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 755/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:52:53 2022] DELETE /identity/v3/projects/032e48429dcb45b682bcef79f0ea27f5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:53:03.070462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=62 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:53:03.698785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:31.871037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.871672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.871672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:31.873028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:31.878145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac22e-dfd8-4a21-b8e6-be19bf391ac0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:31.878601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac22e-dfd8-4a21-b8e6-be19bf391ac0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:31.878704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac22e-dfd8-4a21-b8e6-be19bf391ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.095000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-951ac22e-dfd8-4a21-b8e6-be19bf391ac0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:32.096263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-951ac22e-dfd8-4a21-b8e6-be19bf391ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:32.218433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:32.376093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:32.380556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:32.512383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d19af-b066-43be-9242-79d9e7850350 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:32.518424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:32.519064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:32.519343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:32.519618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:32.527840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:32.529497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d866d-283b-4a82-ad59-2346df220267 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:32.538753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 757/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:33.369337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810bdb-6cef-4967-9c26-c871bc5c644c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.369937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810bdb-6cef-4967-9c26-c871bc5c644c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.369937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810bdb-6cef-4967-9c26-c871bc5c644c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.371295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:33.377302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b84d-5b3a-4a14-ac96-e893b2f24568 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.378074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b84d-5b3a-4a14-ac96-e893b2f24568 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.378265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8b84d-5b3a-4a14-ac96-e893b2f24568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.402906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e8d8b84d-5b3a-4a14-ac96-e893b2f24568 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:33.404152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e8d8b84d-5b3a-4a14-ac96-e893b2f24568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:33.412828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:33.565435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06810bdb-6cef-4967-9c26-c871bc5c644c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:33.581641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06810bdb-6cef-4967-9c26-c871bc5c644c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:33.589605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:33.590251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.590524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.590788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.600424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:33.602038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711ee57-968c-4146-84f1-1554c2503f2e service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:33.611930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 758/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:33.613115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6136d7-5ead-4149-ada0-0190dbc8b5c1 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.613406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6136d7-5ead-4149-ada0-0190dbc8b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.613714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6136d7-5ead-4149-ada0-0190dbc8b5c1 None None] PATH_INFO: `/v3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.615124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 759/1517] 10.222.0.65 () {54 vars in 925 bytes} [Thu Jul 14 10:54:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:54:33.625462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091fa2-be47-4b26-a982-608d12137bba None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:33.625783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091fa2-be47-4b26-a982-608d12137bba None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:33.626063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091fa2-be47-4b26-a982-608d12137bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:33.689623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-20091fa2-be47-4b26-a982-608d12137bba None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:33.690846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-20091fa2-be47-4b26-a982-608d12137bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:33.710045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:33.778830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 759/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jul 14 10:54:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:37.642168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6337-1cdf-4ddf-b6da-da842b3da080 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.642786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6337-1cdf-4ddf-b6da-da842b3da080 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.642786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6337-1cdf-4ddf-b6da-da842b3da080 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.644947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 760/1519] 10.222.0.27 () {54 vars in 960 bytes} [Thu Jul 14 10:54:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:37.653783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebbf3-d78e-4b73-b583-25c72feec96c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.654288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebbf3-d78e-4b73-b583-25c72feec96c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.654392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ebbf3-d78e-4b73-b583-25c72feec96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.680475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8e2ebbf3-d78e-4b73-b583-25c72feec96c None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:37.682009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8e2ebbf3-d78e-4b73-b583-25c72feec96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:37.693436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 760/1520] 10.222.0.27 () {58 vars in 1057 bytes} [Thu Jul 14 10:54:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:37.702007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca6337-1cdf-4ddf-b6da-da842b3da080 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:37.723813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca6337-1cdf-4ddf-b6da-da842b3da080 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:37.732501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:37.733295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:37.733570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:37.733843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:37.743488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:37.745147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ae107-28f1-421e-b638-c46a2d9305ab service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:37.755671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 761/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:44.875297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800f5d-cc01-4aa0-a76c-85f6615cdf2c None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.875858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800f5d-cc01-4aa0-a76c-85f6615cdf2c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.875858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800f5d-cc01-4aa0-a76c-85f6615cdf2c None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.877277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:54:44.883388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f5-47aa-47eb-9ece-36b72aa8a2ff None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:44.884118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f5-47aa-47eb-9ece-36b72aa8a2ff None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:44.884355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc88f5-47aa-47eb-9ece-36b72aa8a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:44.918185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e4fc88f5-47aa-47eb-9ece-36b72aa8a2ff None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:44.919379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e4fc88f5-47aa-47eb-9ece-36b72aa8a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:44.929591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:45.097648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e800f5d-cc01-4aa0-a76c-85f6615cdf2c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:45.119040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e800f5d-cc01-4aa0-a76c-85f6615cdf2c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:45.129940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:45.130827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.131173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.131523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.144916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:45.146773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d8f2f-a3d8-42a1-a33d-69eeb617940d service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:45.156825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 762/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:45.965573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b1bc-9864-4194-ad0f-287008fcb971 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.966310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b1bc-9864-4194-ad0f-287008fcb971 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.966546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533b1bc-9864-4194-ad0f-287008fcb971 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.968921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:54:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:45.977243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57065d-b06d-4f80-86f9-2bf36169ca6c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:45.977693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57065d-b06d-4f80-86f9-2bf36169ca6c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:45.977809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57065d-b06d-4f80-86f9-2bf36169ca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:45.998911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fb57065d-b06d-4f80-86f9-2bf36169ca6c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:46.000166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fb57065d-b06d-4f80-86f9-2bf36169ca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:46.009955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:46.162570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b1bc-9864-4194-ad0f-287008fcb971 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:46.182101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533b1bc-9864-4194-ad0f-287008fcb971 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:46.188337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:46.188983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:46.189267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:46.189582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:46.198697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:46.200329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d55c93-ad56-417c-9d39-6ef873ac83c6 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:46.210261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 764/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:50.257437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.258029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.258029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.259279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:50.265926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1663f-6486-4be9-abb2-38a7e2f594d3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.269662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1663f-6486-4be9-abb2-38a7e2f594d3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.269662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1663f-6486-4be9-abb2-38a7e2f594d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.290412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4ff1663f-6486-4be9-abb2-38a7e2f594d3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:50.291640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4ff1663f-6486-4be9-abb2-38a7e2f594d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:50.300490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:50.378780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:50.380936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:50.466045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7e0f1-dca5-42c7-911c-7c3134235e9b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:50.470142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:50.470785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:50.471062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:50.471332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:50.481281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:50.482755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7d76-7467-42b6-8daa-06b49ebf96c9 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:50.492353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 765/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:52.495386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383fa5-25de-4a4b-b137-1860e95a3bdf None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.496016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383fa5-25de-4a4b-b137-1860e95a3bdf None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.496016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383fa5-25de-4a4b-b137-1860e95a3bdf None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.497363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:52.502716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20abf-fa63-4724-9c61-cf6aa929474e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.503176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20abf-fa63-4724-9c61-cf6aa929474e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.503269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20abf-fa63-4724-9c61-cf6aa929474e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.522595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bcf20abf-fa63-4724-9c61-cf6aa929474e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:52.523787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bcf20abf-fa63-4724-9c61-cf6aa929474e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:52.533222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:52.572391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383fa5-25de-4a4b-b137-1860e95a3bdf None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:52.586306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383fa5-25de-4a4b-b137-1860e95a3bdf None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:52.590713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:52.591359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:52.591645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:52.592169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:52.602913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:52.604440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5bc95-c42b-4291-ba74-52a123b8fbf9 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:52.614920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 767/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:56.285291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.286200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.286200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.288461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:54:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:56.294727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2f3-d0fb-41b2-a1ac-e8bd65332ef3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.295494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2f3-d0fb-41b2-a1ac-e8bd65332ef3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.296153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4f2f3-d0fb-41b2-a1ac-e8bd65332ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.307013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=81 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:56.324302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-83c4f2f3-d0fb-41b2-a1ac-e8bd65332ef3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:56.326092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-83c4f2f3-d0fb-41b2-a1ac-e8bd65332ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:56.336206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:56.660229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:56.663172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:56.735901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:54:56.749619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f60ff5-9c49-4595-a648-7a3bc611410d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:56.753444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:56.754102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:56.754365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:56.754636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:56.764218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:56.765960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f2c55-ee6d-473d-bd06-cb4b6136fc5e service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:56.775320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 768/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:54:58.836080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.836912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.836912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.839045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:54:58.844761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63db2bb-4032-4e29-ae41-3035fb299291 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:58.845218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63db2bb-4032-4e29-ae41-3035fb299291 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:58.845322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63db2bb-4032-4e29-ae41-3035fb299291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:58.868034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c63db2bb-4032-4e29-ae41-3035fb299291 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:54:58.869216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c63db2bb-4032-4e29-ae41-3035fb299291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:58.879266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:54:59.182160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:54:59.185347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:54:59.201056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c99ea4-9cbf-4cb1-88d4-29f4a0ff651f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:54:59.205663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:54:59.206309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:54:59.206590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:54:59.206866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:54:59.216843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:54:59.218386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deadd06-0226-4011-b246-300f48116218 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:54:59.228581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 770/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:54:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:55:08.862996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=11 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:55:09.223987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=8 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:28.619014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:28.620067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:28.620513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:28.645177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:28.646719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:28.656063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 770/1540] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:28.665858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:28.667499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:28.667499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:28.694648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:28.700963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:28.714489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 771/1541] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:28.720213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:28.738945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e690353-d72c-4bee-ae38-289a9764e651 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:28.748718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:28.749673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:28.749959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:28.750227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:28.751005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:28.751311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:28.753011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:28.778043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 771/1542] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:28.794014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:28.805590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891cd4b-094a-4888-8dff-24208cfcaf1a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:28.816298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:28.816986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:28.817265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:28.817540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:28.818261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:28.824587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffdf1e-52b9-430d-a5f1-5de31406dbbf admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:28.882141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 772/1543] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:28 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:28.887550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:28.901180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c53656-2081-4f42-b2b9-31f6436eaeb9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:28.990066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:28.990751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:28.991029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:28.991300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:28.992015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:28.993928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.007147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74272196-60db-42b1-afb2-281a91e38877 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.007508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74272196-60db-42b1-afb2-281a91e38877 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.007710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74272196-60db-42b1-afb2-281a91e38877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.030161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 772/1544] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:28 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:29.035248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.040909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-74272196-60db-42b1-afb2-281a91e38877 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.042113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-74272196-60db-42b1-afb2-281a91e38877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.051051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0c74-2d6d-419f-a4a1-8a4da93d57e3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.060606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.061654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.062167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.062659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.063006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 773/1545] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:29.063977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.065915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.071863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154ccc-bf08-4f2e-b1a4-0c12e0a5532e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.072175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154ccc-bf08-4f2e-b1a4-0c12e0a5532e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.072443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154ccc-bf08-4f2e-b1a4-0c12e0a5532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.085623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 773/1546] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:56:29.102044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.121286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc8e34-4557-4cea-aba4-ee9ccdc6f2d5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.125469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-bd154ccc-bf08-4f2e-b1a4-0c12e0a5532e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.126867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bd154ccc-bf08-4f2e-b1a4-0c12e0a5532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.136660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.137734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.138237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.138713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.139894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.142770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d15b9-904e-4f29-9757-7b4be97d3c8f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.161131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 774/1547] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:29.162455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.162739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.162954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 774/1548] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:29.163073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.164672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.165244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.165748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.197470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.198679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.216020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.216668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 775/1549] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.217324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.217494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.235008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783fb13-9ae0-4f43-bb87-4b51f8f168df None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.243937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.244625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.244939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.245182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.247114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.247424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.249608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de47654-32d0-46a5-b3fe-2122a1ed806c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.259306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 775/1550] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.260639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.274463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 776/1551] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.276108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fabc-fce3-4678-b851-5388413adb56 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.276402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fabc-fce3-4678-b851-5388413adb56 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.276682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9fabc-fce3-4678-b851-5388413adb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.277129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa259add-f58e-43ae-8442-1dbb83b14313 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.286114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.288271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.290392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.290671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] PATH_INFO: `/v3/projects/a3160540122441ce970560d7a3db2521/users/6ead76e4f3564d6eb614bb911ccaf674/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.303827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-18f9fabc-fce3-4678-b851-5388413adb56 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.305139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-18f9fabc-fce3-4678-b851-5388413adb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.307053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3160540122441ce970560d7a3db2521, user_id=6ead76e4f3564d6eb614bb911ccaf674, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.309801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be348-28fd-4f23-834e-cfebbf7cd0fe admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.313377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 777/1552] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.314417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dd740-1360-4cbc-ab2a-70526493a13f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.314676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dd740-1360-4cbc-ab2a-70526493a13f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.314957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dd740-1360-4cbc-ab2a-70526493a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.325541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 776/1553] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:29 2022] PUT /identity/v3/projects/a3160540122441ce970560d7a3db2521/users/6ead76e4f3564d6eb614bb911ccaf674/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:29.327107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.327391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.327670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.333825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-447dd740-1360-4cbc-ab2a-70526493a13f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.335032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-447dd740-1360-4cbc-ab2a-70526493a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.344997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.346192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.397250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 778/1554] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.398678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb64-d079-40f7-85f1-29c28ad5355c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.398974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb64-d079-40f7-85f1-29c28ad5355c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.399262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701fb64-d079-40f7-85f1-29c28ad5355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.409339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 777/1555] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.410657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.421235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4701fb64-d079-40f7-85f1-29c28ad5355c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.421690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6073151-277a-4587-b5fd-4ba7b358f75e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.422443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4701fb64-d079-40f7-85f1-29c28ad5355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.426311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.426958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.427242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.427519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.428247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.430161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886db1c8-aaa1-4525-8a0e-676ea3fe7b2a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.430460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 779/1556] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.431732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.432051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.432327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.450934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.452449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.458810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 778/1557] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:29.460386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992a21-f9e1-49ce-9080-30f8bbef4461 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.460669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992a21-f9e1-49ce-9080-30f8bbef4461 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.467784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1992a21-f9e1-49ce-9080-30f8bbef4461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.467987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 780/1558] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.467987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.472131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aaed8-8ff6-497b-ac7e-83322d891177 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.477371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:29.484905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d1992a21-f9e1-49ce-9080-30f8bbef4461 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.486105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d1992a21-f9e1-49ce-9080-30f8bbef4461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.528969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.529802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.530150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.530426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.531171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.531731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.533228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf3e3cd-1e90-434a-baa8-9a978468428a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.537587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:29.546242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 781/1559] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.547550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.547840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.548163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.550757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 779/1560] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.552188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555d5d-6827-425c-b56a-8f1290526e9f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.552484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555d5d-6827-425c-b56a-8f1290526e9f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.552765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36555d5d-6827-425c-b56a-8f1290526e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.572020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-36555d5d-6827-425c-b56a-8f1290526e9f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.573237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-36555d5d-6827-425c-b56a-8f1290526e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.577016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.578276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.582411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 780/1561] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.583669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315696e-9d94-433a-9254-221750896535 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.583993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315696e-9d94-433a-9254-221750896535 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.584277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315696e-9d94-433a-9254-221750896535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.586782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 782/1562] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.587781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.600962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775c188-66e0-459c-9cb2-aa88a5496772 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.604554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b315696e-9d94-433a-9254-221750896535 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.605751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b315696e-9d94-433a-9254-221750896535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.606939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.607641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.607989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.608271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.609052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.609359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.610974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9feff-eed6-490a-8eff-84e37e39ecca admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.614085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 781/1563] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.614844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315696e-9d94-433a-9254-221750896535 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.624794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 783/1564] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.626756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79f65-c6fb-4863-af42-d8c3a36994bc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.627043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79f65-c6fb-4863-af42-d8c3a36994bc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.627316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e79f65-c6fb-4863-af42-d8c3a36994bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.634214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315696e-9d94-433a-9254-221750896535 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.638947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.639623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.639934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.640543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.641249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.642267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:29.642267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:56:29.642267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:56:29.643481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.646016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d5e79f65-c6fb-4863-af42-d8c3a36994bc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.651160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d5e79f65-c6fb-4863-af42-d8c3a36994bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.655974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 784/1565] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.657020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.657303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.657614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.668679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 782/1566] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:29.669601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.678319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:29.679517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:29.679649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b7846-1592-4340-a711-1eb438702a22 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.684325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.684964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.685246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.685517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.686323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.688066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 785/1567] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:29.688548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.688872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.698209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5c30-e5a6-4f48-8c2e-913cc5594af0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.702965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.703605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.704042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.704317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.705295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.705629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.707405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.719528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 783/1568] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:29.720625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.724313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 786/1569] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.725043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.735800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e70a4-f32e-493b-ab10-4e32e28acdef admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.781045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038d42-e282-4d36-a986-f969e0792c77 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.785121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.785775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.786053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.786425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.796702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.797363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.797670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.797944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.798670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.798976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.800639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.815069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 787/1570] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.816252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.828347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581c1e9-6413-45fb-a835-90a45a19a5b4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.834073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.834760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.835060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.835328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.836051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.837076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:29.837076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:56:29.837076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:56:29.838273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.858966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.860526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.869849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 788/1571] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:29.870622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 784/1572] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:29.870959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.871504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.882486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10f714-689f-47c9-9edb-64b53f38c427 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.919297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:29.945002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:29.952178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.953604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.954245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.954638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.955492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.957762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.961363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76b648-1618-481c-8b00-76409da4a3fc admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.966950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.968446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:29.968732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:29.969021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:29.969760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:29.970071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:29.971902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:29.973888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 785/1573] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:29.975766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.986858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 789/1574] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:29.987707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:29.989162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3b986-11bb-4307-884c-c5bd0197ecf6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:29.998722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:29.999587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.000294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.000294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.000933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:30.001237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.002369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:30.002369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:56:30.002369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:56:30.003288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.003912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360dbf9-5c5d-43f9-9bd9-b7a8576961fd admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.009306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.009979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.010753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.011040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.011914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.013928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.018787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 786/1575] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:30.019673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.037255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7216-df0a-420c-99a6-c071d566f736 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.041769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 790/1576] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:29 2022] POST /identity/v3/users => generated 295 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.042749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.045392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.046370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.047096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.047576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.048551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.051449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.054687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2c47f-bf4f-4c1f-a6c0-42509fd24c69 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.061245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.062547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.063067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.063278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.064104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.066705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.088959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 787/1577] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/projects => generated 393 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.089741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.107118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 791/1578] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.107885 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.167291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379545a-7964-4de0-9cb1-e3a6a303f0dc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.172111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.172785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.173062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.173331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.174028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.175959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.176291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7448b36-add3-4ded-a41f-21bdbb716313 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.181279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.182300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.182788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.183255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.184211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.186298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.199059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 792/1579] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.205136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 788/1580] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.205136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.206298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.213202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cc0ec-b180-489e-87a4-030a54d6ddb2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.216600 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c648f4e-d6b9-4bad-9cc8-52bc49a4ae20 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.218238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.219009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.219428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.219704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.220513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.221529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.222184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.222490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.222738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.222841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.223427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.225459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.256876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 789/1581] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.258078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.271291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 793/1582] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.276120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.338218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b11c-aa31-444a-b110-682ae2b6486a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.352718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:30.356920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30729392-cf4a-492f-96e5-e18150cb0d96 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.363723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.364443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.364715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.365116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.365890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.367920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.378022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:30.391828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 794/1583] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.392669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.405812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646285b7-864b-4e5d-8fdf-c47d22fe88d0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.410900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.411703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.412075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.412296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.412998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.415477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.418162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.419004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.419447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.419905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.420910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.422694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.437906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 795/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.437906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.438504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 790/1585] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.439763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.449955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eefadc-30a8-459d-a6a8-7b17fdc1a38b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.450842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b5fb6-9f34-4398-aba9-af2681230e88 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.454782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.455685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.456000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.456285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] PATH_INFO: `/v3/projects/9ea25562cd9b44dfa599d3a8c3b1f324/users/240c18e4ef65400eb55abde15424b819/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.456821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.457651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.458071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.458496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.459220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.461021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.475065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 791/1586] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.475065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.475973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea25562cd9b44dfa599d3a8c3b1f324, user_id=240c18e4ef65400eb55abde15424b819, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.477074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:30.477074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:56:30.477074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:56:30.479751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.485197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41498491-d97d-439c-b2ec-ceffdcb1ae11 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.490729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.491691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.491691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.492000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.492698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.494634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.507900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 796/1587] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:30 2022] PUT /identity/v3/projects/9ea25562cd9b44dfa599d3a8c3b1f324/users/240c18e4ef65400eb55abde15424b819/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:30.507900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.525960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 792/1588] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.525960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.535806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c6df3-09a4-47cc-965e-44a1822b7041 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.587784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bc3ba-7e33-4007-9b9e-e1f316360432 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.592962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.593958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.594270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.594596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.595293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.597580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.602899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.603626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.603940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.604224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.604994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.606697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.625569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 793/1589] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.625569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.635726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 797/1590] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:30.636608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.641054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a734-cfd6-415f-aabd-003a56da5d67 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.645807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.646340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.646626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.646955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.647759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.649487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.660939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 794/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.661557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.668122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946a1f8-6eb5-44f0-b03d-0b7e17d47d22 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.674905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd68e86-a06d-43d7-8c9d-4a356ccefc26 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.677525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.679032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.679410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.679756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.680730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.681001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.681453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.681738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.682062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.682808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.683268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8f6d2-1108-46c8-bdd5-9f26ac2c14ce admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.684717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.702258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 798/1592] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.702506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 795/1593] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.705823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.705986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.705986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.705986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.718650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66f565-f160-47aa-b634-d79f69bad57f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.729568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.730733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.731131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.731326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.732128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.734935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.736159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] MFA Rules not processed for user `240c18e4ef65400eb55abde15424b819`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:30.742560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:30.752024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 796/1594] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.752024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.763312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a019d03-2b2b-4331-94c3-683e81c6096d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.770690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.771590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.772046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.772444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.773610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.775690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.789039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 797/1595] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.789985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.801506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b1ab4-6cf9-4a49-8c1a-f6f5e561f758 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.806805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.807792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.808172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.808695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.809555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.811717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.819306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 799/1596] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:30.819306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.830547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072378a-45da-4c39-b8f6-078f34403ec2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.836951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.837807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.839060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.839060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] PATH_INFO: `/v3/projects/2ff1f327753b4708b0940964aa3d761c/users/3e4012911dad42edbffead00e0fd55e6/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.848930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 798/1597] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:30.848930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.854544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:30.861355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff1f327753b4708b0940964aa3d761c, user_id=3e4012911dad42edbffead00e0fd55e6, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.865045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.881856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d6dbb-52f0-4c70-97f5-10970f7144a9 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.891146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 800/1598] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:30 2022] PUT /identity/v3/projects/2ff1f327753b4708b0940964aa3d761c/users/3e4012911dad42edbffead00e0fd55e6/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:30.894832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:30.944998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:30.970232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.972579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.973322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.973790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] PATH_INFO: `/v3/projects/bee9d276e8894710aa906773baa327e3/users/a99ca7f13e29411c91b606f59e635121/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.987674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5f0c6-783e-43ed-95aa-9cbb3402a48d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:30.993093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:30.993924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:30.994302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:30.994637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:30.995441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:30.997506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:30.998495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee9d276e8894710aa906773baa327e3, user_id=a99ca7f13e29411c91b606f59e635121, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.002673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.009674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 801/1599] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:31.011178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.022791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328913-52c9-494e-8acd-a1d4e3a2b0b1 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.028732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 799/1600] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:30 2022] PUT /identity/v3/projects/bee9d276e8894710aa906773baa327e3/users/a99ca7f13e29411c91b606f59e635121/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:31.030337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.103197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.106435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.106734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.107112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.107975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.110739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c0813-5d14-465c-a2f0-cbc7052e35a5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.118734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5b26f-1102-4798-a9a0-1469af5f5155 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.124588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.125973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.126546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.126881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] PATH_INFO: `/v3/projects/c2d5301114244be2a74210484c6536ef/users/b47991b6880c4a91ab42d2567eee61aa/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.128038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 802/1601] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:31.129763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51990f0-4a47-4098-9394-cd28c6c99d73 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.130092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51990f0-4a47-4098-9394-cd28c6c99d73 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.130336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51990f0-4a47-4098-9394-cd28c6c99d73 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.131922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 803/1602] 10.222.0.27 () {54 vars in 961 bytes} [Thu Jul 14 10:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:31.132788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51990f0-4a47-4098-9394-cd28c6c99d73 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.153155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d5301114244be2a74210484c6536ef, user_id=b47991b6880c4a91ab42d2567eee61aa, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.157665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51990f0-4a47-4098-9394-cd28c6c99d73 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.159792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6f959-1f6a-456f-be7c-b0206afab33e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.166864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.167655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.167894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.168185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.169067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.171159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.181904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 800/1603] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:31 2022] PUT /identity/v3/projects/c2d5301114244be2a74210484c6536ef/users/b47991b6880c4a91ab42d2567eee61aa/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:31.183794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 804/1604] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:31.184959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.187275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6225a7-992b-4440-a118-032d77cafcc0 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.187575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6225a7-992b-4440-a118-032d77cafcc0 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.187857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6225a7-992b-4440-a118-032d77cafcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.205433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757c711-a0fa-47b1-9690-ed1b5a966a0c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.208559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0a6225a7-992b-4440-a118-032d77cafcc0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.209838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0a6225a7-992b-4440-a118-032d77cafcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.281101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.282004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.282284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.282558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] PATH_INFO: `/v3/projects/5dcd3b32ea75463c95bb2d451c59cdd1/users/5cc60574eb7f4df6b193c244973ed8c3/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.298339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 801/1605] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.299832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069a3b8-0452-4481-89ec-b7ee1bea5a92 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.300152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069a3b8-0452-4481-89ec-b7ee1bea5a92 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.300433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069a3b8-0452-4481-89ec-b7ee1bea5a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.304450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dcd3b32ea75463c95bb2d451c59cdd1, user_id=5cc60574eb7f4df6b193c244973ed8c3, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.307434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.310249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:31.337195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 805/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:31 2022] PUT /identity/v3/projects/5dcd3b32ea75463c95bb2d451c59cdd1/users/5cc60574eb7f4df6b193c244973ed8c3/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:31.338128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.345102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-1069a3b8-0452-4481-89ec-b7ee1bea5a92 None None] MFA Rules not processed for user `a99ca7f13e29411c91b606f59e635121`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.346144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1069a3b8-0452-4481-89ec-b7ee1bea5a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.389553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:31.414234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a463fa-6012-4e8a-9228-89a2f24cdd67 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.432941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.433868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.434456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.434456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] PATH_INFO: `/v3/projects/a20518b41d7b4867a89281d8aad64a96/users/4521b5a2287d40a18ce9d0f17124c706/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.455425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a20518b41d7b4867a89281d8aad64a96, user_id=4521b5a2287d40a18ce9d0f17124c706, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.458513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a4aea-6934-45fa-888d-ab7286e4645a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.461257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 802/1607] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.462760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6615-59ad-4190-9655-99276133c661 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.463041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6615-59ad-4190-9655-99276133c661 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.463321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6615-59ad-4190-9655-99276133c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.488909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 806/1608] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:31 2022] PUT /identity/v3/projects/a20518b41d7b4867a89281d8aad64a96/users/4521b5a2287d40a18ce9d0f17124c706/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:31.491082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.491390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.491672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.517396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.520042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.521548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d90e6615-59ad-4190-9655-99276133c661 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.522735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d90e6615-59ad-4190-9655-99276133c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.606201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 803/1609] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.606896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6615-59ad-4190-9655-99276133c661 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.610301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 807/1610] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.611164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.612749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.632186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6615-59ad-4190-9655-99276133c661 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.637272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.638289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.639068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.639068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] PATH_INFO: `/v3/projects/b8f93ae8626a4048a18e0e5934cb692c/users/0c401e40f59a4abfbd62435e3dd08c38/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.665807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f93ae8626a4048a18e0e5934cb692c, user_id=0c401e40f59a4abfbd62435e3dd08c38, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.674875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2371f6-59a7-4df7-ba56-a0597c790314 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.800509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 804/1611] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:31 2022] PUT /identity/v3/projects/b8f93ae8626a4048a18e0e5934cb692c/users/0c401e40f59a4abfbd62435e3dd08c38/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:31.800509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f74c-ce83-4cce-894d-3e9dcdefd210 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.800509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f74c-ce83-4cce-894d-3e9dcdefd210 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.800509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9f74c-ce83-4cce-894d-3e9dcdefd210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.801981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf3e48-1f44-4eb3-81a2-fa4cde751fdb None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.802516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fcd9f74c-ce83-4cce-894d-3e9dcdefd210 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.802516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fcd9f74c-ce83-4cce-894d-3e9dcdefd210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.802675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:31.829610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.830268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.830550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.830824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.839188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 805/1612] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.841003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258cd4-6872-464f-971d-e9e9b184e9aa None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.841509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258cd4-6872-464f-971d-e9e9b184e9aa None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.842029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258cd4-6872-464f-971d-e9e9b184e9aa None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.844126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 806/1613] 10.222.0.27 () {54 vars in 961 bytes} [Thu Jul 14 10:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:31.845325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.845678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.846021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.846729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:31.848245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:31.859158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 808/1614] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:31.860211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.875497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] MFA Rules not processed for user `4521b5a2287d40a18ce9d0f17124c706`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:31.876814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:31.900333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:31.961609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5f89-5b87-49d1-bfcc-5bad45ce8ef2 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:31.974431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:31.975492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:31.976043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:31.976486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:31.985598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 807/1615] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:31.986889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:31.999305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd485c-d9cd-4f35-8058-9348068a1da1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.004386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.005971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.006544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.007569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.021941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.025285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c534b3-2621-4218-861d-ecf85c07318a service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.046778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 808/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:32.050794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.051968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.052638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.066714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.069382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a901b-7553-4882-9797-e49aeaf8174e service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.087127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 809/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:32.088502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.088854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.089192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.090933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] MFA Rules not processed for user `0c401e40f59a4abfbd62435e3dd08c38`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:32.093525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:32.115665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:32.117114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:32.133813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 810/1618] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:32.145829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.160359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4dc3e-29b0-4401-8f60-2577dead78b0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.165195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.166190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.166594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.166990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.184146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.186973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.199260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 811/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:32.200835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.204798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 809/1620] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:32.206183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.215655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82207b-8105-4417-80f4-34ba2e7d8598 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.224032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b906b-8289-4d24-b116-f0dd950ec2f1 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.227615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.231012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.231287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.231520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.232262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.237520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.237645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.238992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.239556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.240119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.303146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 812/1621] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:32.304009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.395021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:32.400955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.402877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.424226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 810/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:32.425253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.449407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec631ea-3475-4adc-97e7-809e6b2fa8e0 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.460262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.461254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.461688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.462125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.463028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.469498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.469831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddb89f-d418-4a7d-8861-ba78e3b36e69 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.481684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.482363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.482755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.483224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.483952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.485883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.512982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 811/1623] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/users => generated 296 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:32.517273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.536280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee325-69c1-45dd-95dc-4a833a6ef0a2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.548696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 813/1624] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:32.551353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.553354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.553496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.554924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.555384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.556331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.560828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.588278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 812/1625] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:56:32.594308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.618699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984635ec-bd2a-453a-b13b-5bc319be5e83 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.629519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:32.694116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51fb5-2832-4616-aafa-b82828430d86 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.699613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.701089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.701873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.702478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.703811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.708307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d3aa6-ea75-4dde-81ce-be976727306b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.738200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 814/1626] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:56:32.740683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e02f-159a-423d-b6d8-d828b4c0d6cc None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.741090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e02f-159a-423d-b6d8-d828b4c0d6cc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.741432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e02f-159a-423d-b6d8-d828b4c0d6cc None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.742914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 815/1627] 10.222.0.27 () {54 vars in 912 bytes} [Thu Jul 14 10:56:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:32.752723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.752960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e02f-159a-423d-b6d8-d828b4c0d6cc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:32.756498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.757412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.757684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] PATH_INFO: `/v3/projects/701967a165d0496583002af9652204ee/users/b8b36886205f479a91efedd7322d38fb/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.771711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e02f-159a-423d-b6d8-d828b4c0d6cc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:32.778673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:32.779926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.780482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.781172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.781703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.783593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9d014-cf20-4537-b9cf-49ecb7b8bc08 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.797866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 816/1628] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:32.799414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:32.799763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:32.800211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:32.806507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=701967a165d0496583002af9652204ee, user_id=b8b36886205f479a91efedd7322d38fb, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:32.809191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:32.823383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:32.824984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:32.867373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 813/1629] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:32 2022] PUT /identity/v3/projects/701967a165d0496583002af9652204ee/users/b8b36886205f479a91efedd7322d38fb/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:32.876849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.037994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 817/1630] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:33.039722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.056389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f7e1-1410-4282-8d52-b9fe977519e1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.061855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.062708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.063101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.063741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.064783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.066756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.070959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee0a4b-8395-476b-9fac-a4b3a3b6a16d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.081196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 818/1631] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:33.084059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.085032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.085316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.085636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] PATH_INFO: `/v3/projects/0443e73ba59f4ad183ceac162f1810fb/users/112a8186681c450cb75f1e55f867b7d6/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.087909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.122914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0443e73ba59f4ad183ceac162f1810fb, user_id=112a8186681c450cb75f1e55f867b7d6, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.129891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389739-a118-438a-b582-d6d458bae31c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.136266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:33.166648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 814/1632] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:32 2022] PUT /identity/v3/projects/0443e73ba59f4ad183ceac162f1810fb/users/112a8186681c450cb75f1e55f867b7d6/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:33.174868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb02988-a90c-46ec-a271-0f0de2eb85ca None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.174868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb02988-a90c-46ec-a271-0f0de2eb85ca None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.175284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb02988-a90c-46ec-a271-0f0de2eb85ca None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.177941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 815/1633] 10.222.0.27 () {54 vars in 912 bytes} [Thu Jul 14 10:56:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:33.180999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb02988-a90c-46ec-a271-0f0de2eb85ca None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.218821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249ebdd-3d06-4d6c-9948-09eb0e3487a4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.329663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb02988-a90c-46ec-a271-0f0de2eb85ca None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.338619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.340142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.340747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.341368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.343862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.345329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.346542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.346883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] PATH_INFO: `/v3/projects/701967a165d0496583002af9652204ee/users/b8b36886205f479a91efedd7322d38fb/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.352679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=701967a165d0496583002af9652204ee, user_id=b8b36886205f479a91efedd7322d38fb, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.360130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.386734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 816/1634] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:33 2022] PUT /identity/v3/projects/701967a165d0496583002af9652204ee/users/b8b36886205f479a91efedd7322d38fb/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:33.397569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.411656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:33.511054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.514230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5a04b-aa1a-4d65-93af-a2f3b5c3568b service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.547530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 819/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:33.549741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.551305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.551858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.555860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958452-7ecd-45ae-beaa-1b9ffb0a69fb admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.572247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.573924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.574307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.576413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.581191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.587046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724aed34-899a-41de-bd9e-62ff0103c647 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.602783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:33.605106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:33.617859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 817/1636] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:33.617859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.617859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.617859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.678882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] MFA Rules not processed for user `b8b36886205f479a91efedd7322d38fb`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:33.681740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:33.777601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 820/1637] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:33.782111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.798692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:33.809679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabceb5-29f5-462e-a028-f6671b8159be None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:33.822470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:33.823705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.824284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.825966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] PATH_INFO: `/v3/projects/0443e73ba59f4ad183ceac162f1810fb/users/112a8186681c450cb75f1e55f867b7d6/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.836748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0443e73ba59f4ad183ceac162f1810fb, user_id=112a8186681c450cb75f1e55f867b7d6, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:33.840853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e2f7d-fba7-40b3-8d06-3b419b9fd714 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:33.849339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 818/1638] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:33.851598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:33.869808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 821/1639] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:33 2022] PUT /identity/v3/projects/0443e73ba59f4ad183ceac162f1810fb/users/112a8186681c450cb75f1e55f867b7d6/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:33.879938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:33.880502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:33.880984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:33.951153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] MFA Rules not processed for user `112a8186681c450cb75f1e55f867b7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:33.957717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:33.970367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc99136-940c-4ae2-97f0-59fa15359329 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:34.094857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:34.116562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:34.117240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:34.117525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:34.123968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:34.139416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 822/1640] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:34.295300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:34.296804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:34.329209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 819/1641] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:34.478485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:34.497156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947abca-46c0-46d0-b78e-bf9d0536ecd0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:34.509702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:34.510966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:34.511254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:34.511533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:34.600864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:34.615648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614722-0553-4228-9fd4-7cbcb5712114 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:34.622418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:34.623077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:34.623451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:34.623904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:34.642401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:34.643854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:34.672339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:34.672590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 820/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:34.674145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:34.687248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 823/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:35.034590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:35.051182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4f7d6-f1b3-4ac5-88c1-87ab14664512 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:35.051917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:35.067104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:35.068758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:35.070442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:35.071699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:35.207750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c479ef-857f-4115-b289-82b9ab2cac79 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:35.220476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:35.223264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:35.227809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:35.228409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] PATH_INFO: `/v3/users/a99ca7f13e29411c91b606f59e635121` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:35.233110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99ca7f13e29411c91b606f59e635121)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:35.235019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: WARNING py.warnings [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:35.235019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: warnings.warn(msg) Jul 14 10:56:35.235019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: Jul 14 10:56:35.238733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:35.272739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:35.309094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:35.310729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7da2a1-f47d-4c93-91a4-0c692ea16ff2 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:35.338030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 821/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:35.415158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99ca7f13e29411c91b606f59e635121'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:35.463625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99ca7f13e29411c91b606f59e635121'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:35.464522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99ca7f13e29411c91b606f59e635121'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:35.497644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99ca7f13e29411c91b606f59e635121'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:35.513206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99ca7f13e29411c91b606f59e635121'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:35.540712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 824/1645] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:35 2022] DELETE /identity/v3/users/a99ca7f13e29411c91b606f59e635121 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:56:35.545383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:35.545930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:35.546398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:35.590143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:35.595646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:35.818466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 822/1646] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:35.877290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:35.879056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:35.913933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:35.915677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:35.946752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac278d2e-7a93-4d06-ad9b-3a43e2c84959 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:35.962846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:35.964274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:35.964786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:35.965261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] PATH_INFO: `/v3/users/0c401e40f59a4abfbd62435e3dd08c38` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:35.969283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c401e40f59a4abfbd62435e3dd08c38)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:35.972906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:35.993494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:36.060429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94144dbb-6c1f-465f-b2c6-4c549707591e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:36.074900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:36.075575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:36.075850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:36.076151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:36.096465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c401e40f59a4abfbd62435e3dd08c38'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:36.106227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:36.107724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea421-9631-49c3-a267-e951d638d958 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:36.114681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c401e40f59a4abfbd62435e3dd08c38'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:36.117578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c401e40f59a4abfbd62435e3dd08c38'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:36.131790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c401e40f59a4abfbd62435e3dd08c38'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:36.134936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 825/1647] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:36.146007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c401e40f59a4abfbd62435e3dd08c38'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:36.158912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 823/1648] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:35 2022] DELETE /identity/v3/users/0c401e40f59a4abfbd62435e3dd08c38 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:36.165248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:36.165533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:36.165816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:36.201319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:36.202749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:36.378396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 826/1649] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:36.404747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:36.406205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:36.534125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f114d-cdb2-447b-bca8-de4cd423dc83 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:36.574995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:36.575667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:36.575991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:36.576304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:36.595504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:36.597060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:36.619264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 824/1650] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:56:37.646343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:37.648489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:37.707311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1daa76-a86a-45f8-b774-d7eb187632be None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:37.726950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:37.729016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:37.731562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:37.731947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] PATH_INFO: `/v3/projects/b8f93ae8626a4048a18e0e5934cb692c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:37.739663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f93ae8626a4048a18e0e5934cb692c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:37.741785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:37.858197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:37.928713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f93ae8626a4048a18e0e5934cb692c', 'request_id': 'req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:37.958559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f93ae8626a4048a18e0e5934cb692c', 'request_id': 'req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:37.980874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f93ae8626a4048a18e0e5934cb692c', 'request_id': 'req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.004876 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] The token cache is being invalidate because project b8f93ae8626a4048a18e0e5934cb692c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:38.005143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.006609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.017065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 827/1651] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:37 2022] DELETE /identity/v3/projects/b8f93ae8626a4048a18e0e5934cb692c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:56:38.018557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:38.030077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cdf2-d82c-409f-a626-b41fece03211 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:38.114660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:38.116722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:38.236126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:38.239444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:38.240600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:38.243838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] PATH_INFO: `/v3/users/112a8186681c450cb75f1e55f867b7d6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:38.246497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=112a8186681c450cb75f1e55f867b7d6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:38.250771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:38.321581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c1c5-1cda-4bfe-92da-5e8a7122a34c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:38.333042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:38.335458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:38.336678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:38.337306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:38.337614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] PATH_INFO: `/v3/users/240c18e4ef65400eb55abde15424b819` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:38.339582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=240c18e4ef65400eb55abde15424b819)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:38.344844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:38.368015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112a8186681c450cb75f1e55f867b7d6'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.400197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112a8186681c450cb75f1e55f867b7d6'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.401288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112a8186681c450cb75f1e55f867b7d6'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.414572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112a8186681c450cb75f1e55f867b7d6'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.436705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-06a45b23-1320-46ae-b8ed-7236eb55d97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112a8186681c450cb75f1e55f867b7d6'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.450161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '240c18e4ef65400eb55abde15424b819'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.452645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 825/1652] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:38 2022] DELETE /identity/v3/users/112a8186681c450cb75f1e55f867b7d6 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:38.475109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '240c18e4ef65400eb55abde15424b819'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.476300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '240c18e4ef65400eb55abde15424b819'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.485519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '240c18e4ef65400eb55abde15424b819'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.499193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '240c18e4ef65400eb55abde15424b819'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:38.508718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 828/1653] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:38 2022] DELETE /identity/v3/users/240c18e4ef65400eb55abde15424b819 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:38.514409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:38.514947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:38.515747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:38.544055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:38.545761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:38.674265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 826/1654] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:38.694113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:38.696259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:38.744053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:38.768736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a78673-86ff-45d5-915e-fe8b4d773f11 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:38.772704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:38.773586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:38.773908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:38.774195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:38.786706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:38.786706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd1d55-a622-4523-8100-326b14e6ea56 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:38.799079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 829/1655] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:39.037155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:39.039216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:39.062023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a8fe-f1a0-47c1-ab0d-58836884d436 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:39.076135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:39.077234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:39.077570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:39.077848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] PATH_INFO: `/v3/projects/bee9d276e8894710aa906773baa327e3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:39.079931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee9d276e8894710aa906773baa327e3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:39.081178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: WARNING py.warnings [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:56:39.081178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: warnings.warn(msg) Jul 14 10:56:39.081178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: Jul 14 10:56:39.085805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:39.233325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee9d276e8894710aa906773baa327e3', 'request_id': 'req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:39.258806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee9d276e8894710aa906773baa327e3', 'request_id': 'req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:39.310504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee9d276e8894710aa906773baa327e3', 'request_id': 'req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:39.339370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] The token cache is being invalidate because project bee9d276e8894710aa906773baa327e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:39.339660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:39.340364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f0cc99d1-da7b-41e7-95dc-bdb89845eb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:39.345362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 827/1656] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:39 2022] DELETE /identity/v3/projects/bee9d276e8894710aa906773baa327e3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:39.350052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:39.350343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:39.350640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:39.394972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:39.396317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:39.597385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 830/1657] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:39.607087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:39.607386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:39.607664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:39.635224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:39.637383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:39.713745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 828/1658] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:39.718335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:39.750916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df3b5d-6a4d-4774-8508-b6cf433d7cf5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:39.766788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:39.767476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:39.768020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:39.768463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:39.769359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:39.769887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:39.771756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:39.808259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 831/1659] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:56:39.813169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:39.837798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3359-985c-4640-88db-c66bc1b36cd8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:39.850761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:39.851473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:39.851758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:39.852066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:39.862571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:39.873015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:39.929194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:39.955132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:39.956693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 829/1660] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:39 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:39.990494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac03ab-a040-433d-867c-6ae85b192e82 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.143370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.144127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.145725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.145857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.148691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.150732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.199917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 832/1661] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:39 2022] POST /identity/v3/users => generated 302 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:56:40.200850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:40.212204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac246298-9325-4ec0-b0cd-b412fed8ea3d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.233239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.234440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.234895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.235238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.236019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.237672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.270369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 830/1662] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:40.271752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:40.290835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce04942-ba67-4824-8926-6009fcbe0da8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.297067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.298348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.298641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.298920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.299827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.301619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0e74e-71a5-49fb-891b-c05e0b2267b7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.315110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 833/1663] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:40.319067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:40.333637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6455-57a3-4f3d-9866-e697b457dbdc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.355312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.356793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.357261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.358743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] PATH_INFO: `/v3/projects/e80594acad6a42f795f3c76c1b26cd82/users/d135844c524041c7afd698717f87a385/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.407378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80594acad6a42f795f3c76c1b26cd82, user_id=d135844c524041c7afd698717f87a385, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.411340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.434589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 831/1664] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:40 2022] PUT /identity/v3/projects/e80594acad6a42f795f3c76c1b26cd82/users/d135844c524041c7afd698717f87a385/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:40.449765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.450102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.450385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.524503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] MFA Rules not processed for user `d135844c524041c7afd698717f87a385`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:40.525964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:40.609095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:40.684973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:40.687162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:40.690068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 834/1665] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:40.694177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:40.848705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468248c2-e8a3-41bc-8386-3cb61301697d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.855643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.856678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.857100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.858176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.859213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.861580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.879138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adfa5-6b5f-42cc-a653-ebc33f7a4bf4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:40.890933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:40.891839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:40.892162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:40.892938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] PATH_INFO: `/v3/projects/0443e73ba59f4ad183ceac162f1810fb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:40.896736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0443e73ba59f4ad183ceac162f1810fb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:40.898742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:40.941320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 835/1666] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:40 2022] POST /identity/v3/projects => generated 405 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:56:40.946120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.018617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0443e73ba59f4ad183ceac162f1810fb', 'request_id': 'req-1d6640a1-1264-46d9-aee4-beb2532dad59'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:41.032947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0443e73ba59f4ad183ceac162f1810fb', 'request_id': 'req-1d6640a1-1264-46d9-aee4-beb2532dad59'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:41.043981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88f498-1951-4772-bd02-05da1afa9a62 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.045394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0443e73ba59f4ad183ceac162f1810fb', 'request_id': 'req-1d6640a1-1264-46d9-aee4-beb2532dad59'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:41.075204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] The token cache is being invalidate because project 0443e73ba59f4ad183ceac162f1810fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:41.075492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:41.077819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1d6640a1-1264-46d9-aee4-beb2532dad59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:41.081430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 832/1667] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:40 2022] DELETE /identity/v3/projects/0443e73ba59f4ad183ceac162f1810fb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:41.086086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507979bb-cba7-4bf7-8699-2db6a8ff3078 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.086378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507979bb-cba7-4bf7-8699-2db6a8ff3078 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.086641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507979bb-cba7-4bf7-8699-2db6a8ff3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.136864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-507979bb-cba7-4bf7-8699-2db6a8ff3078 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:41.138104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-507979bb-cba7-4bf7-8699-2db6a8ff3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:41.192821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.194470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.194941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.195358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.196295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.201996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.216706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:41.241378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 836/1668] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:40 2022] POST /identity/v3/users => generated 309 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:41.246326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.266960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:41.268895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:41.290746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ab5dc-78e8-459f-8093-f660305f3077 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.293271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 833/1669] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:41.296606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.297605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.301774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.302268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.302845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.305672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.306065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.307276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.308976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.323982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 837/1670] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:41.327734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:41.329341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:41.329580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.339747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 834/1671] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:41.344135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c682e-169c-45e3-8797-ee81cd9ec597 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.344677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.350491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.351457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.351804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.352202 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] PATH_INFO: `/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9/users/b1c6a13c057e4553b782b59b470c114e/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.356528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc422c5-e9a7-4f09-89e6-10b7201e8d3d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.379174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.379893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.380205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.380480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.381226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:41.381568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.383208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.393964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ad053f98254cc2aa6e442f9cdbf2d9, user_id=b1c6a13c057e4553b782b59b470c114e, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.398433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.425605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 838/1672] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:41 2022] PUT /identity/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9/users/b1c6a13c057e4553b782b59b470c114e/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:41.426345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 835/1673] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:41.432993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.434250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.464806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d33cd-ba4f-424b-b812-3ae6acdbe7da admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.542894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3e6dd-89b9-43f6-8fac-aba2c9d2cf75 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.550814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.554521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.556105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.556105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.556105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.560077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.601490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.602334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.602690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.603465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.604338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.606646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.618861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 839/1674] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:41 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:41.623532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.629405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 836/1675] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:41.630457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.642501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e59d-baeb-4c94-ae32-790519985339 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.726632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.727544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.727920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.728178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] PATH_INFO: `/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9/users/b1c6a13c057e4553b782b59b470c114e/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.730704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ad053f98254cc2aa6e442f9cdbf2d9, user_id=b1c6a13c057e4553b782b59b470c114e, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.736515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffec84d-d97b-4a1c-869d-ca1d16ba159b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.764714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 837/1676] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:41 2022] PUT /identity/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9/users/b1c6a13c057e4553b782b59b470c114e/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:41.769479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.769771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.770037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.784448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d8d44-d1be-4baa-8fa4-0e8eb92f4c7d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:41.825639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] MFA Rules not processed for user `b1c6a13c057e4553b782b59b470c114e`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:41.829995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:41.907014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:41.927534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:41.928460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:41.930231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:41.930603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:41.931184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:41.932723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:41.940278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:41.957810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 838/1677] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:41.969832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:41.973725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:41.993487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 840/1678] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:41 2022] POST /identity/v3/users => generated 282 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:56:41.999974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:42.035178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f4320-24d5-44b7-888f-7b8d0c168408 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.049809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:42.050707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.051245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.051638 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.053032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:42.057143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:42.086562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 841/1679] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:42.091102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:42.120472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87689a2-cd14-49c7-989a-b49cec0a5834 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.145646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:42.146820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.147325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.147607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.148375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:42.149677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9e77-5566-4fa6-b0fd-25c8b10a0ffb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.153096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:42.173274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:42.173948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.174226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.174498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.202496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 842/1680] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:42.203514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:42.205633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:42.209405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7c98-2021-4521-b5dd-774c539f6d9e service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:42.236383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ff184-dd61-466f-9277-9d887ef9a2b1 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.247910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 839/1681] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:42.268973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:42.269948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.270229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.270583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] PATH_INFO: `/v3/projects/0327922b992d492f81dc4227e23109be/users/cdc7fbc7a6d64f10a82fd7c344af244c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.319433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0327922b992d492f81dc4227e23109be, user_id=cdc7fbc7a6d64f10a82fd7c344af244c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:42.331573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:42.367517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 843/1682] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:42 2022] PUT /identity/v3/projects/0327922b992d492f81dc4227e23109be/users/cdc7fbc7a6d64f10a82fd7c344af244c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:42.379147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715cbdf-4111-43b7-88ae-bd86bef4667c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.379605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715cbdf-4111-43b7-88ae-bd86bef4667c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.379780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715cbdf-4111-43b7-88ae-bd86bef4667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.421601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:42.423791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:42.452884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3715cbdf-4111-43b7-88ae-bd86bef4667c None None] MFA Rules not processed for user `cdc7fbc7a6d64f10a82fd7c344af244c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:42.460276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3715cbdf-4111-43b7-88ae-bd86bef4667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:42.586968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaee7b-3eec-4cfe-a9de-348cce79a412 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.594543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:42.595795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.596191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.596651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] PATH_INFO: `/v3/projects/9ea25562cd9b44dfa599d3a8c3b1f324` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.604276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea25562cd9b44dfa599d3a8c3b1f324)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:42.606651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:42.620204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 840/1683] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:42.620204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.620204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.620204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.674331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] MFA Rules not processed for user `5cc60574eb7f4df6b193c244973ed8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:42.676106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:42.710675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea25562cd9b44dfa599d3a8c3b1f324', 'request_id': 'req-a343c8a0-8762-4d44-af62-a3c0f9642654'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:42.719992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea25562cd9b44dfa599d3a8c3b1f324', 'request_id': 'req-a343c8a0-8762-4d44-af62-a3c0f9642654'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:42.728633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea25562cd9b44dfa599d3a8c3b1f324', 'request_id': 'req-a343c8a0-8762-4d44-af62-a3c0f9642654'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:42.751111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] The token cache is being invalidate because project 9ea25562cd9b44dfa599d3a8c3b1f324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:42.751408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:42.752121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a343c8a0-8762-4d44-af62-a3c0f9642654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:42.754165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 844/1684] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:42 2022] DELETE /identity/v3/projects/9ea25562cd9b44dfa599d3a8c3b1f324 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:42.755814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f97571-6679-44f6-8827-4a3bdc339102 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.756161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f97571-6679-44f6-8827-4a3bdc339102 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.756439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f97571-6679-44f6-8827-4a3bdc339102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.821432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-19f97571-6679-44f6-8827-4a3bdc339102 None None] MFA Rules not processed for user `6ead76e4f3564d6eb614bb911ccaf674`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:42.823456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-19f97571-6679-44f6-8827-4a3bdc339102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:42.833585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 841/1685] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:42.834348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:42.837401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:42.843977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:42.872653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293ecb-d558-4c08-b7fb-f32eb4813b88 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:42.891487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:42.950550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 845/1686] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:42.952785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:42.953533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:42.953903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:42.979494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:42.980923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.003226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.003956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.004240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.004520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.005350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.007170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeebfc8-bf26-451a-be95-8eff7dc82743 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.065602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 842/1687] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:42 2022] POST /identity/v3/projects => generated 363 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:43.071717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.072048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.074412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.114277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 846/1688] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:43.115403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.117254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.125886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] MFA Rules not processed for user `5cc60574eb7f4df6b193c244973ed8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:43.128165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.254313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 843/1689] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:43.255157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.275809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ea86e-786b-4d9f-9560-b246ae9fb1cc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.434944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bc0ed-7cd4-446d-b6f4-f6797609efcb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.438247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:43.447599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.449157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.449654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.451224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.452119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.460525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26669d-8c86-40c3-9e0c-1a437bd504ae admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.478598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.479311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.479710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.480123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.529635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 844/1690] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:43 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:43.533581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.533924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.534078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.604429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:43.605994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.629047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 845/1691] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:43.630375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.632682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.683047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b8888-0ad0-4cf7-b629-ec0e817a67a4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.688927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.693713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.704680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.705468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.705822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.706170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.716198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 847/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:43.723680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.725192 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.734504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.739388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:43.754870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 846/1693] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:43.755914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.774861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ac78f-8690-4d52-9616-8bc67f621c69 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.775547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f9fe3-2e00-4335-8698-f7dafc969ef7 service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.785950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.786945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.787051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.787446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.790081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.793091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.793801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.794210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.794785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.795067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] PATH_INFO: `/v3/users/d135844c524041c7afd698717f87a385` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.797111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d135844c524041c7afd698717f87a385)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.803846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.816045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 847/1694] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:43.819489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.827697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:43.842642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f87c-2fa1-44fa-8c12-ee5d45f386f3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.852964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.853783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.854129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.854472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:43.855383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:43.855649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:43.857805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:43.905913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 848/1695] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:43.910227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:43.955309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110903c-9373-4cc9-b66f-fb9a0b13ea36 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:43.955693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd135844c524041c7afd698717f87a385'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:43.971968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:43.972671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:43.972953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:43.973229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] PATH_INFO: `/v3/projects/418584fdc5104b719ca9e3b9430e722d/users/af1012b4c1734a43a153f5c35e5234a5/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.002254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd135844c524041c7afd698717f87a385'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:44.004197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd135844c524041c7afd698717f87a385'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:44.024597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=418584fdc5104b719ca9e3b9430e722d, user_id=af1012b4c1734a43a153f5c35e5234a5, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.027277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.041446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd135844c524041c7afd698717f87a385'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:44.064589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 849/1696] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:43 2022] PUT /identity/v3/projects/418584fdc5104b719ca9e3b9430e722d/users/af1012b4c1734a43a153f5c35e5234a5/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:44.064589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.080228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd135844c524041c7afd698717f87a385'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:44.096562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 848/1697] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:43 2022] DELETE /identity/v3/users/d135844c524041c7afd698717f87a385 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:44.097774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.100458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.223386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15300ca-0d08-4ac2-ba53-3f43e7a40d0c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.256130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34d70b-d204-4b08-8894-54c5da776188 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.268882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.269848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.270133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.270513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.271401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.282114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610ac03-ee20-40e5-8873-8b4636ec313a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.290850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:44.306859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 849/1698] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:44.308030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.310497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.310968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.343583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:44.344995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.355304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 850/1699] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:44.356336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.369206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.369946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.370227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.370515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.371444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.380210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.399905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b07c9-0b97-412e-a303-b68cba164398 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.415709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.418630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.419125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.420099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] PATH_INFO: `/v3/projects/418584fdc5104b719ca9e3b9430e722d/users/af1012b4c1734a43a153f5c35e5234a5/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.424434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418584fdc5104b719ca9e3b9430e722d, user_id=af1012b4c1734a43a153f5c35e5234a5, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.427817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2afe1-c244-4d2b-bebb-33a6d825f137 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.447014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 851/1700] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:44 2022] PUT /identity/v3/projects/418584fdc5104b719ca9e3b9430e722d/users/af1012b4c1734a43a153f5c35e5234a5/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:44.460117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.460837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.461324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.463006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 850/1701] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:44 2022] POST /identity/v3/projects => generated 371 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:44.469645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.476576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.520059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] MFA Rules not processed for user `af1012b4c1734a43a153f5c35e5234a5`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:44.521509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.628655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:44.635413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8a4ef-b6a1-4063-8eb4-b9582a32ad62 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.657336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.658077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.659071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.661121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.661831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.667885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.719731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 851/1702] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:44 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:44.719731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.719731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.762448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 852/1703] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:44.763865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.787157 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b29c-600e-42f5-8f92-62dd586aabc1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.803517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.804615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.805013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.805499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.806436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.812613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.840534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 853/1704] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:44.845734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.855697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:44.890469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33451c9-042b-4db2-b786-be1bbf7c6e9f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:44.901557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:44.902242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:44.902525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:44.902804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:44.932687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:44.934231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:44.969924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 852/1705] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:44.973377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:44.998725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3f494-ca4d-4a35-8a41-f8dbb23ae486 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.011128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:45.012185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.012468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.012739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.013518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:45.015250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:45.017012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317cf94-4533-41b6-a738-2eeafa0933f1 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.027745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:45.028615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.028997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.029375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.047367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 853/1706] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:45.050919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:45.055916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734882-3903-4c89-a1b1-102ed1d398a8 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:45.059667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:45.079512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 854/1707] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:45.086697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5b5e-81f0-4455-a4d4-577900363095 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.102330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:45.104400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.104690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.107564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] PATH_INFO: `/v3/projects/e0fcfa39b02e4e9ea773e8c23c3b9551/users/5623e80e405b43c28404f32071b8e67e/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.159729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fcfa39b02e4e9ea773e8c23c3b9551, user_id=5623e80e405b43c28404f32071b8e67e, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:45.170267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:45.201373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 854/1708] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:45 2022] PUT /identity/v3/projects/e0fcfa39b02e4e9ea773e8c23c3b9551/users/5623e80e405b43c28404f32071b8e67e/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:45.211849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.212776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.213407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.216956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:45.318070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] MFA Rules not processed for user `5623e80e405b43c28404f32071b8e67e`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:45.320942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:45.379017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb023f-faaa-4417-8ec3-758f8825bf5b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.398759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:45.399361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.399643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.399960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.408845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:45.440426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:45.483398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 855/1709] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:45.489062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:45.586558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:45.588091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:45.610288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 855/1710] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:45.693628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eb11a-bccf-47bb-b736-ddae53703f6c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.705565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:45.706777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.708093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.708494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.709323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:45.715720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa73ee-6cd8-4b02-8e5e-38a6a7bdeabc admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:45.776736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:45.777266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 856/1711] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:45 2022] POST /identity/v3/projects => generated 373 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:45.798067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95691f-b433-48f0-b9ef-aa641a8f6180 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:45.806734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:45.936919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1888e-3aeb-4c75-bf88-0b23983ae005 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.937158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1888e-3aeb-4c75-bf88-0b23983ae005 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.937437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1888e-3aeb-4c75-bf88-0b23983ae005 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:45.939151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 857/1712] 10.222.0.27 () {54 vars in 912 bytes} [Thu Jul 14 10:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:56:45.956142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:45.956487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:45.956770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.008935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.011158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.011158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.011158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.011158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.012906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.082748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 856/1713] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:45 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:46.086879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:46.104409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:46.111591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:46.130378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab436eb8-51c7-4563-9221-69b676268fee admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:46.154822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.155907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.156047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.156303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.157129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.165793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.200507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 857/1714] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:46.201551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:46.230058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150454c-e0ee-4db0-a56a-7910fef4a8e6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:46.249975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.250482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.250764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.251037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] PATH_INFO: `/v3/projects/da6a71d345fc4d73800c548291810983/users/6b8e030eb42f43d8bcfbe0a1189120cc/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.286833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6a71d345fc4d73800c548291810983, user_id=6b8e030eb42f43d8bcfbe0a1189120cc, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.289648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.338768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 858/1715] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:46 2022] PUT /identity/v3/projects/da6a71d345fc4d73800c548291810983/users/6b8e030eb42f43d8bcfbe0a1189120cc/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:46.345354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:46.359362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 858/1716] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:56:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 14 10:56:46.486322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9262-a733-48d5-8791-c00b36e6896f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:46.502436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.503130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.503596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.504059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.504933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.507052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.521222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 859/1717] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:46.526337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:46.546499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:46.590516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e5c0-8299-4e99-8e64-88fd849aeff5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:46.613227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.614155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.624673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.625005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] PATH_INFO: `/v3/projects/da6a71d345fc4d73800c548291810983/users/6b8e030eb42f43d8bcfbe0a1189120cc/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.638234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6a71d345fc4d73800c548291810983, user_id=6b8e030eb42f43d8bcfbe0a1189120cc, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.669125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebe9ba-41a0-48a0-a10c-17b5f4eb34bb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.734093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 859/1718] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:46 2022] PUT /identity/v3/projects/da6a71d345fc4d73800c548291810983/users/6b8e030eb42f43d8bcfbe0a1189120cc/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:46.737475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8ca4-9c48-47a7-8841-31af4e7b913c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:46.745863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.746150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.746426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.760791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:46.850617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] MFA Rules not processed for user `6b8e030eb42f43d8bcfbe0a1189120cc`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:46.852990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:46.887437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:46.891099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:46.894230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:46.894748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:46.922084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:46.923474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:46.947548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:46.951186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 860/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:47.018440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 860/1720] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:47.031421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:47.197119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305f88b-ea0b-4a8c-be96-71b03875a106 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:47.204118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:47.208215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:47.208823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:47.208823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:47.291987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:47.295462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:47.377123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 861/1721] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:47.466835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:47.473918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:47.762325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:47.786990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25681d-374a-48a2-953e-311bd2bf242d service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:47.787473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3df26-ebe3-4b81-a87e-a25304c6a146 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:47.804509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:47.806579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:47.807152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:47.809632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:47.814892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:47.818652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:47.818939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:47.828738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] PATH_INFO: `/v3/projects/e80594acad6a42f795f3c76c1b26cd82` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:47.835604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80594acad6a42f795f3c76c1b26cd82)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:47.840635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:48.033838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80594acad6a42f795f3c76c1b26cd82', 'request_id': 'req-b86868bb-22c8-47e8-9275-ddcad399d0a9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.051949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80594acad6a42f795f3c76c1b26cd82', 'request_id': 'req-b86868bb-22c8-47e8-9275-ddcad399d0a9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.061459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:48.067309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:48.074690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80594acad6a42f795f3c76c1b26cd82', 'request_id': 'req-b86868bb-22c8-47e8-9275-ddcad399d0a9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.102549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] The token cache is being invalidate because project e80594acad6a42f795f3c76c1b26cd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:48.102842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.103652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.111253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 861/1722] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:47 2022] DELETE /identity/v3/projects/e80594acad6a42f795f3c76c1b26cd82 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:48.115108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:48.115550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 862/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:48.117227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:48.375533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:48.377704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:48.399701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86868bb-22c8-47e8-9275-ddcad399d0a9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:48.407300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce071c2a-0526-4242-849b-de3f1fb8c186 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:48.422768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:48.423815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:48.424366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:48.424837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] PATH_INFO: `/v3/users/4521b5a2287d40a18ce9d0f17124c706` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:48.425958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:48.426498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4521b5a2287d40a18ce9d0f17124c706)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:48.426928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:48.429302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:48.429612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] PATH_INFO: `/v3/users/b1c6a13c057e4553b782b59b470c114e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:48.432790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:48.432906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c6a13c057e4553b782b59b470c114e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:48.456198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:48.529240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:48.571031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4521b5a2287d40a18ce9d0f17124c706'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.585123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:48.618630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4521b5a2287d40a18ce9d0f17124c706'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.619668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4521b5a2287d40a18ce9d0f17124c706'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.633201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c6a13c057e4553b782b59b470c114e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.651206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4521b5a2287d40a18ce9d0f17124c706'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.668534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-481a3bae-ae44-4350-9e67-d613feea1c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4521b5a2287d40a18ce9d0f17124c706'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.685073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c6a13c057e4553b782b59b470c114e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.685902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 863/1724] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:48 2022] DELETE /identity/v3/users/4521b5a2287d40a18ce9d0f17124c706 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:56:48.686630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c6a13c057e4553b782b59b470c114e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.690334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:48.692228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:48.692714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:48.712888 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c6a13c057e4553b782b59b470c114e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.730476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7b3dcc95-f131-4197-924e-1bd0a2dc3af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c6a13c057e4553b782b59b470c114e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:48.745402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 862/1725] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:48 2022] DELETE /identity/v3/users/b1c6a13c057e4553b782b59b470c114e => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:48.747545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:48.747545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:48.747545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:48.777100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] MFA Rules not processed for user `3e4012911dad42edbffead00e0fd55e6`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:48.781610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:48.800685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:48.801918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:48.968885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 864/1726] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:48.985062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:48.987067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:49.019944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 863/1727] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:49.106573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:49.108333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:49.228956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e02340-c2d4-4016-8bb1-ffd30ae160c8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:49.244721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:49.251112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.251483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.256481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.260111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bc431-b2f7-4c28-93b1-b55c8fd34e10 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:49.269462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:49.270340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.270762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.271175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.289147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:49.292901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:49.294222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:49.301755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb52f7-b345-470f-95d1-4674c46653e2 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:49.325236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 864/1728] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:56:49.327176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69cfb6-48df-43b6-be82-0522b019237b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.327176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69cfb6-48df-43b6-be82-0522b019237b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.327655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69cfb6-48df-43b6-be82-0522b019237b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.329415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 865/1729] 10.222.0.27 () {54 vars in 960 bytes} [Thu Jul 14 10:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:49.330348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 865/1730] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:49.339677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c954e-1ae6-4c6b-b504-ae2ad21991a2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.340116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c954e-1ae6-4c6b-b504-ae2ad21991a2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.340502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c954e-1ae6-4c6b-b504-ae2ad21991a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.382285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7a8c954e-1ae6-4c6b-b504-ae2ad21991a2 None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:49.383699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7a8c954e-1ae6-4c6b-b504-ae2ad21991a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:49.455752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:49.552464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 866/1731] 10.222.0.27 () {58 vars in 1057 bytes} [Thu Jul 14 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:49.575816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:49.590994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d2428-6222-4c87-9155-b143f21bd75a service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:49.601545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:49.602408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.602848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.603183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.608959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:49.632536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:49.634829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e61676-6862-4ca5-9774-2e798b06b8fb service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:49.652430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 866/1732] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:49.717963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.718248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.718531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:49.779231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] MFA Rules not processed for user `b47991b6880c4a91ab42d2567eee61aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:49.780522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:49.962486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 867/1733] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:49.973566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:49.974486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:49.975076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:50.032081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] MFA Rules not processed for user `b47991b6880c4a91ab42d2567eee61aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:50.040844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:50.072337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 867/1734] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:50.079857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:50.081321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:50.125240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46785-4135-4414-a7e0-5578ee4f6060 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:50.139432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:50.140465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:50.141185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:50.141584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:50.176777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:50.178412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f67cd4-911d-4bcd-9fe8-38a10717caa0 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:50.215293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 868/1735] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:50.364817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:50.366491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:50.408559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc8447-fee6-453f-af13-8d6fcbcfbe1c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:50.422590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:50.423624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:50.424062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:50.426202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] PATH_INFO: `/v3/users/cdc7fbc7a6d64f10a82fd7c344af244c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:50.427834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc7fbc7a6d64f10a82fd7c344af244c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:50.430045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:50.528320 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:50.596194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc7fbc7a6d64f10a82fd7c344af244c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:50.664225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc7fbc7a6d64f10a82fd7c344af244c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:50.667978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc7fbc7a6d64f10a82fd7c344af244c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:50.690446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc7fbc7a6d64f10a82fd7c344af244c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:50.703936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc7fbc7a6d64f10a82fd7c344af244c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:50.715506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 868/1736] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:50 2022] DELETE /identity/v3/users/cdc7fbc7a6d64f10a82fd7c344af244c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:50.720266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:50.720775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:50.720898 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:50.758304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:50.759464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:50.958875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 869/1737] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:50.983701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:50.986121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:51.142630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:51.144265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:51.161950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41320d61-7494-4706-aa34-f89ac5273a55 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:51.185328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02843cb5-89f1-4ef7-a4c5-d2eec2f7ffce admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:51.187462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:51.188750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.189255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.189732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] PATH_INFO: `/v3/projects/a20518b41d7b4867a89281d8aad64a96` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.193597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a20518b41d7b4867a89281d8aad64a96)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:51.196147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:51.204443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:51.205150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.205678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.205678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.250046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:51.251526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:51.271583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 869/1738] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:51.273699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:51.277561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:51.291411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:51.306950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f25a6-d6b4-4899-b94a-78453c9bfbdb service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:51.310949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a20518b41d7b4867a89281d8aad64a96', 'request_id': 'req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.321468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a20518b41d7b4867a89281d8aad64a96', 'request_id': 'req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.341434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a20518b41d7b4867a89281d8aad64a96', 'request_id': 'req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.366554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] The token cache is being invalidate because project a20518b41d7b4867a89281d8aad64a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:51.366954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.367968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.371012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 870/1739] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:51 2022] DELETE /identity/v3/projects/a20518b41d7b4867a89281d8aad64a96 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:51.376495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:51.378539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:51.522100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:51.523058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.523352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.523609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] PATH_INFO: `/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.532304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ad053f98254cc2aa6e442f9cdbf2d9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:51.536321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:51.599719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43158b-4a7a-4828-8c8d-5622b0dd4e75 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:51.609899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:51.611210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.611716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.612216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] PATH_INFO: `/v3/users/b8b36886205f479a91efedd7322d38fb` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.613850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b36886205f479a91efedd7322d38fb)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:51.616078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:51.663800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:51.674507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ad053f98254cc2aa6e442f9cdbf2d9', 'request_id': 'req-3e7fe9aa-0070-4698-8421-fe36d7489ed2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.696140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ad053f98254cc2aa6e442f9cdbf2d9', 'request_id': 'req-3e7fe9aa-0070-4698-8421-fe36d7489ed2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.711069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ad053f98254cc2aa6e442f9cdbf2d9', 'request_id': 'req-3e7fe9aa-0070-4698-8421-fe36d7489ed2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.736221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] The token cache is being invalidate because project 40ad053f98254cc2aa6e442f9cdbf2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:51.736699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.737791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e7fe9aa-0070-4698-8421-fe36d7489ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.740708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 870/1740] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:51 2022] DELETE /identity/v3/projects/40ad053f98254cc2aa6e442f9cdbf2d9 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:51.743334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b5ed-d040-4d4e-91ed-75f2cb43faea None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.744558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b5ed-d040-4d4e-91ed-75f2cb43faea None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.745016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740b5ed-d040-4d4e-91ed-75f2cb43faea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.755577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b36886205f479a91efedd7322d38fb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.794765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c740b5ed-d040-4d4e-91ed-75f2cb43faea None None] MFA Rules not processed for user `6ead76e4f3564d6eb614bb911ccaf674`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:51.798121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c740b5ed-d040-4d4e-91ed-75f2cb43faea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:51.803168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b36886205f479a91efedd7322d38fb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.806466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b36886205f479a91efedd7322d38fb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.824662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b36886205f479a91efedd7322d38fb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.847066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-664410b9-8a1b-4a22-9e60-8d9596b94ca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b36886205f479a91efedd7322d38fb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:51.871592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 871/1741] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:51 2022] DELETE /identity/v3/users/b8b36886205f479a91efedd7322d38fb => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:51.875356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c71e-2e1d-4ba3-adbf-888db9d37354 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.875538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c71e-2e1d-4ba3-adbf-888db9d37354 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.876034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c71e-2e1d-4ba3-adbf-888db9d37354 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.878053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 872/1742] 10.222.0.27 () {54 vars in 912 bytes} [Thu Jul 14 10:56:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:56:51.879417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b630-318a-4d9b-8b96-8c03b8f45de6 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.880228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b630-318a-4d9b-8b96-8c03b8f45de6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.880731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b630-318a-4d9b-8b96-8c03b8f45de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.948962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 871/1743] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:51.950773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:51.951153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:51.951684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:51.952906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8144b630-318a-4d9b-8b96-8c03b8f45de6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:51.958168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8144b630-318a-4d9b-8b96-8c03b8f45de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:52.056563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:52.064923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:52.145284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:52.195341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 873/1744] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:52.197191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.197484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.197759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.249050 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:52.254659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:52.293494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 874/1745] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:56:52.297431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.326126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e984aa-4b1e-40d0-b426-16ef10a00c69 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.333891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 872/1746] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:52.350617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.351306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.351585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.351858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.352664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:52.352968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.355086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.388656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 875/1747] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:52.393142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.421062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a913fc-bf0e-47af-b95b-9902beba1642 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.428713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.432252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.432682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.433096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.433956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.436595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.522009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 873/1748] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:52 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:52.524005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.541261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5bdd-94d6-4992-b13c-6c5211737b99 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.558007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.559847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:52.715306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a912f-be3f-4385-a292-1b916936a43c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.718872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.719723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.721092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.721930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.723797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.730242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.731104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.732045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.733360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.733787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.754175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.756162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.799789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 876/1749] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:52 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:52.803488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 874/1750] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:56:52.805422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.823279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03866f8e-e582-4520-b37b-a36aac069654 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.832833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.834475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.834753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.835040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.835858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.839084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.840837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:52.862349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 875/1751] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:52.866271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.880871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259004d3-7221-4ef8-adb6-1681576e8d94 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.896460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.897721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.898264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.898874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:52.899720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:52.904317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c940b36-89a6-42dc-a914-012cd2632b97 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:52.939723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:52.951660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 877/1752] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:52.955190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b686877-855c-45be-a038-e984e744b472 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:52.972852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:52.974033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:52.974452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:52.974887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] PATH_INFO: `/v3/projects/f41b10ff2d8e49ceae763c34c0a29ae9/users/eee21f7245ee42e2931586462f082196/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.011828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41b10ff2d8e49ceae763c34c0a29ae9, user_id=eee21f7245ee42e2931586462f082196, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:53.014682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:53.031918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 876/1753] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:52 2022] PUT /identity/v3/projects/f41b10ff2d8e49ceae763c34c0a29ae9/users/eee21f7245ee42e2931586462f082196/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:53.043332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:53.043670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:53.044106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.102272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] MFA Rules not processed for user `eee21f7245ee42e2931586462f082196`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:53.103730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:53.264534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 878/1754] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:53.265668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:53.301803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:53.305679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:53.437095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:53.495368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a94ca-26ed-42da-b0df-ea96fbadbef3 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:53.544576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14a94a-7983-4e83-81bb-3ad0802305e6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:53.558568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:53.559278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:53.561944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:53.562273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.563039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:53.578751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:53.582518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:53.587132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:53.589561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:53.590912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] PATH_INFO: `/v3/projects/0327922b992d492f81dc4227e23109be` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.595788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0327922b992d492f81dc4227e23109be)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:53.606712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:53.666407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 877/1755] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:53 2022] POST /identity/v3/projects => generated 387 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:53.668134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:53.674453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:53.782772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0327922b992d492f81dc4227e23109be', 'request_id': 'req-1c24e82d-484e-4b81-ab10-b561f0995aff'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:53.809286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0327922b992d492f81dc4227e23109be', 'request_id': 'req-1c24e82d-484e-4b81-ab10-b561f0995aff'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:53.816059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0327922b992d492f81dc4227e23109be', 'request_id': 'req-1c24e82d-484e-4b81-ab10-b561f0995aff'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:53.831215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] The token cache is being invalidate because project 0327922b992d492f81dc4227e23109be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:53.831743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:53.832719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:53.839505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 879/1756] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:53 2022] DELETE /identity/v3/projects/0327922b992d492f81dc4227e23109be => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:53.841218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:53.843739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:53.854184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e4125-96f3-42c3-b14d-874cd5bf33f5 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:53.956405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e82d-484e-4b81-ab10-b561f0995aff admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:53.963962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:53.965206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:53.965495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:53.965774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.966600 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:53.970128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:53.987983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:53.988887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:53.989175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:53.989453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] PATH_INFO: `/v3/projects/701967a165d0496583002af9652204ee` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:53.991641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=701967a165d0496583002af9652204ee)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:53.995772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.020515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 880/1757] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:53 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:54.021179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:54.026892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.040033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.092176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da2ecf-b512-49cf-8117-1d4fe11f61d9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:54.140793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701967a165d0496583002af9652204ee', 'request_id': 'req-6f018067-4fe8-4615-8e4b-b7f0f9509aba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.155529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701967a165d0496583002af9652204ee', 'request_id': 'req-6f018067-4fe8-4615-8e4b-b7f0f9509aba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.172452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701967a165d0496583002af9652204ee', 'request_id': 'req-6f018067-4fe8-4615-8e4b-b7f0f9509aba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.173932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:54.190288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:54.191494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.192108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.192767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] PATH_INFO: `/v3/users/af1012b4c1734a43a153f5c35e5234a5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.194549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1012b4c1734a43a153f5c35e5234a5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:54.195775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] The token cache is being invalidate because project 701967a165d0496583002af9652204ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:54.196099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.196667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.196772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.204711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 878/1758] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:53 2022] DELETE /identity/v3/projects/701967a165d0496583002af9652204ee => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:54.205735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.214120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.233495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f018067-4fe8-4615-8e4b-b7f0f9509aba admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:54.246699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:54.247406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.247693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.247998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.248812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:54.250492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc3cc3-5194-4bd2-a5d0-7fefc5dd7314 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.281056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 879/1759] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:54.282138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.282415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.282698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.320281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:54.321526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.348371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1012b4c1734a43a153f5c35e5234a5'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.357183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 880/1760] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:54.358229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.382770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4b0c3-8c61-4bf7-bf07-26b4004cab57 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:54.385699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1012b4c1734a43a153f5c35e5234a5'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.386760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1012b4c1734a43a153f5c35e5234a5'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.399060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1012b4c1734a43a153f5c35e5234a5'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.400559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:54.404196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.404474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.405135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] PATH_INFO: `/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd/users/2a17995bfeab40a4a7c2e4da83e094cb/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.412033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-432cd8bf-dbe2-4630-b2b8-d525ec053948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1012b4c1734a43a153f5c35e5234a5'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:54.434145 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 881/1761] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:54 2022] DELETE /identity/v3/users/af1012b4c1734a43a153f5c35e5234a5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:54.438201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554fbd-e1e2-4dec-b56b-26ec4e2a58ae None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.438530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554fbd-e1e2-4dec-b56b-26ec4e2a58ae None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.438807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554fbd-e1e2-4dec-b56b-26ec4e2a58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.444836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ade901ce67742d5ae6d6b564d1ee1bd, user_id=2a17995bfeab40a4a7c2e4da83e094cb, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:54.458122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.488561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-85554fbd-e1e2-4dec-b56b-26ec4e2a58ae None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:54.491815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-85554fbd-e1e2-4dec-b56b-26ec4e2a58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.505800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 881/1762] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:54 2022] PUT /identity/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd/users/2a17995bfeab40a4a7c2e4da83e094cb/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:54.510778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.517004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.692212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 882/1763] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:54.697270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cb296-1f8e-463f-ac6b-b8ae01331f9a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.697562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cb296-1f8e-463f-ac6b-b8ae01331f9a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.697841 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cb296-1f8e-463f-ac6b-b8ae01331f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.775543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-448cb296-1f8e-463f-ac6b-b8ae01331f9a None None] MFA Rules not processed for user `b47991b6880c4a91ab42d2567eee61aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:54.777027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-448cb296-1f8e-463f-ac6b-b8ae01331f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:54.792879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e610c2-d1d4-48e8-b9b1-cc04f126e79a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:54.818130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:54.819790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.820776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.821044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.828184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:54.834046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.867496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 882/1764] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:56:54.874283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.908932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362065c0-a102-46ef-9195-4618756d2035 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:54.929845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:54.933982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 883/1765] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:54.933982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490366-a207-4a91-8e3d-fbb939c75495 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.933982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490366-a207-4a91-8e3d-fbb939c75495 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.933982 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490366-a207-4a91-8e3d-fbb939c75495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.937229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:54.938952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:54.939918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:54.941372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:54.945423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:54.950354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:54.973046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 883/1766] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:54.974322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:54.988869 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd767-f796-490f-ae3e-940f79e47717 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.000385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e4490366-a207-4a91-8e3d-fbb939c75495 None None] MFA Rules not processed for user `3e4012911dad42edbffead00e0fd55e6`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:55.001785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e4490366-a207-4a91-8e3d-fbb939c75495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:55.013153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.014224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.014569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.014916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] PATH_INFO: `/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd/users/2a17995bfeab40a4a7c2e4da83e094cb/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.022803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ade901ce67742d5ae6d6b564d1ee1bd, user_id=2a17995bfeab40a4a7c2e4da83e094cb, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.025769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.030457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:55.061236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 884/1767] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:54 2022] PUT /identity/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd/users/2a17995bfeab40a4a7c2e4da83e094cb/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:55.061236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.115184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:55.165710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 884/1768] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:55.168830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.170014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.170276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.211598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc9bc-b7ff-45a7-b42d-c408ca33d314 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.240072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.240436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] MFA Rules not processed for user `2a17995bfeab40a4a7c2e4da83e094cb`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:55.240902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.242755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.243531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.243643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:55.248195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.255785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.334614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 885/1769] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:55 2022] POST /identity/v3/projects => generated 385 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:55.335848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.417657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 885/1770] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:55.431480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.433101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:55.442796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c152f-e21e-4ea3-b5ad-4eb4e6fbb86c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.460323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.461197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.461637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.462043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.462961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.465662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.517287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 886/1771] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:55 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:55.518766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.533860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5c187-a327-4159-9697-53ac46d1aa46 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.553940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.554871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.555258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.555758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.556656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.558532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.572498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d572b4-9e37-47c3-a8ea-9f46a3c25adc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.583354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 887/1772] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:56:55.584860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.613807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1293fe-d5ad-48e8-8784-6bc93195f1f0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.624355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.626429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.627222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.629112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.630149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.634367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.665047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 888/1773] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:55.666104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.691692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf2c9d-48c5-43fb-842b-0abf11f60469 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:55.714946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.714946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.714946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.714946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] PATH_INFO: `/v3/projects/528b258b867940339e81c2470f1b72f2/users/e23bb718ad6745b0b822fd58613d5430/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.721883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:55.724241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.725088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.725524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.750481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528b258b867940339e81c2470f1b72f2, user_id=e23bb718ad6745b0b822fd58613d5430, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.756257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996c5f0-e5bd-4820-8bb5-16fe5cffff21 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.778334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:55.791384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 889/1774] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:55 2022] PUT /identity/v3/projects/528b258b867940339e81c2470f1b72f2/users/e23bb718ad6745b0b822fd58613d5430/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:55.794322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:55.795058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:55.795058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:55.913504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] MFA Rules not processed for user `e23bb718ad6745b0b822fd58613d5430`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:55.917480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:55.941780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:55.947017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:55.985736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 886/1775] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:56:55.990517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:55.996979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:56.036817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29ed97-8dd8-45fc-9744-a1bc40ae35a2 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.061770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.171249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 890/1776] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:56.172871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:56.258780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.260107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.260477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.263117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] PATH_INFO: `/v3/projects/418584fdc5104b719ca9e3b9430e722d` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.269438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] RBAC: Authorizing `identity:delete_project(project_id=418584fdc5104b719ca9e3b9430e722d)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.273970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:56.347178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85ddc7-f003-4398-a3e1-67ff88a12c38 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.355913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.359738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.360156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.360510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] PATH_INFO: `/v3/users/eee21f7245ee42e2931586462f082196` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.364229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee21f7245ee42e2931586462f082196)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.371180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:56.388387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418584fdc5104b719ca9e3b9430e722d', 'request_id': 'req-1f322861-3a4e-4d2a-8554-449d44f9854a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.400365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418584fdc5104b719ca9e3b9430e722d', 'request_id': 'req-1f322861-3a4e-4d2a-8554-449d44f9854a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.408033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418584fdc5104b719ca9e3b9430e722d', 'request_id': 'req-1f322861-3a4e-4d2a-8554-449d44f9854a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.428474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] The token cache is being invalidate because project 418584fdc5104b719ca9e3b9430e722d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:56:56.428859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.429708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.431796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 887/1777] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:56:55 2022] DELETE /identity/v3/projects/418584fdc5104b719ca9e3b9430e722d => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:56.432625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:56.436666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:56.490136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee21f7245ee42e2931586462f082196'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.537061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee21f7245ee42e2931586462f082196'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.537926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee21f7245ee42e2931586462f082196'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.567526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee21f7245ee42e2931586462f082196'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.582570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-283089ab-6290-4ef5-9a2e-3f026b5c2e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee21f7245ee42e2931586462f082196'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:56:56.588706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.600369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 891/1778] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:56:56 2022] DELETE /identity/v3/users/eee21f7245ee42e2931586462f082196 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:56.609112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c903-e3a7-4cd7-a9c3-64f68fe49d84 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.609691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c903-e3a7-4cd7-a9c3-64f68fe49d84 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.610211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7c903-e3a7-4cd7-a9c3-64f68fe49d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.639432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f322861-3a4e-4d2a-8554-449d44f9854a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:56.755690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a7c7c903-e3a7-4cd7-a9c3-64f68fe49d84 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:56.758126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a7c7c903-e3a7-4cd7-a9c3-64f68fe49d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:56.829822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:56.846353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:56.847603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.847903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.848200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.849028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:56.850997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fe8b2-74e6-40ba-80f6-270f41f44542 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:56.949872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 888/1779] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/projects => generated 387 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:56.951813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.959521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.959521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:56.960677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 892/1780] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:56.963508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ffc6f-85c6-4046-aac8-b58d743a09ae None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:56.964041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ffc6f-85c6-4046-aac8-b58d743a09ae None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:56.965182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ffc6f-85c6-4046-aac8-b58d743a09ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.040762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:57.042343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.066140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6d8ffc6f-85c6-4046-aac8-b58d743a09ae None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:57.072959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6d8ffc6f-85c6-4046-aac8-b58d743a09ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.286341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 889/1781] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:57.287299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.307920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.335545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8684d-13d1-48e1-b931-697181bd5316 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.348479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 893/1782] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:57.349829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.350122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.350399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.352402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.353253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.353624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.354025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.354843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.361329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.402654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:57.404392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.414720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 894/1783] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:57.415679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.417007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.436521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 890/1784] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:57 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:57.448042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.453238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:57.622078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cee421-73a3-4b1d-9a7b-6b387163beaa admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.632009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.632946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.633349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.633817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.656073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.657735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.665498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e3126-8cca-46e6-8bda-90299d471051 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.683269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.683887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 891/1785] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:56:57.684004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.684283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.684558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.685112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.718801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f0dc-0fa0-4c40-80d5-5ba2e4609549 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.757829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.761407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.761712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.761993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.763071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:56:57.763556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.781629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.783465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:57.803168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:57.817010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 892/1786] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:56:57.817977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.828835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.834371 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.846577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b33c2-8f67-477a-8d4d-588d753ad95c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.856210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 895/1787] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:56:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:56:57.857237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.871529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.872471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.872979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.873433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.877588 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.886215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.887140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9332dc-d3ca-4206-b944-9a18c62b0169 service swift] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:57.897476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:57.898087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:57.898369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:57.898678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:57.904339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:57.906810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:57.959974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 893/1788] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:57.963152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:57.979098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 896/1789] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:57 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:56:57.992669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:58.003573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef863-eddd-4dc3-a749-c1fe5dbde232 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.346934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.352768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21b0a1-6c34-41b8-919a-51e72da14567 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.362896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.363298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.363700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] PATH_INFO: `/v3/projects/be6357215c1646169a897b22529b055d/users/d74f77cb251e4207bdcaf9e6db5f7f31/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.380533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.381823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.382173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.382653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.383435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.398977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:58.465852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6357215c1646169a897b22529b055d, user_id=d74f77cb251e4207bdcaf9e6db5f7f31, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.486059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:58.491003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 897/1790] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:56:57 2022] POST /identity/v3/users => generated 285 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:56:58.492187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:58.547696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30604c4-5e00-462e-bbf9-20583278001d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.589602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 894/1791] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:57 2022] PUT /identity/v3/projects/be6357215c1646169a897b22529b055d/users/d74f77cb251e4207bdcaf9e6db5f7f31/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:56:58.598366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:58.776966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.779917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.780692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.782159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.796492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.798996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:58.849505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d927-6cd6-4724-99f6-75640bb8402e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.864616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.865884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.866327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.866741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.867841 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.869736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:58.882471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 898/1792] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:58.886552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 895/1793] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:58.886965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:58.892857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:58.928613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8710-d68a-43c4-bf70-9aaff211408b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.953706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8b8d9-1825-4d37-8780-53d4ba7bf4f6 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:58.959415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.960491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.961029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.961709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.963085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.968896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:58.972095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:58.972235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d40fa-5c42-4668-ba5c-15e73dc0fbb6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:58.972327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:58.972327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] PATH_INFO: `/v3/projects/be6357215c1646169a897b22529b055d/users/d74f77cb251e4207bdcaf9e6db5f7f31/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:58.980242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6357215c1646169a897b22529b055d, user_id=d74f77cb251e4207bdcaf9e6db5f7f31, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:58.988863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:59.027030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 896/1794] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:58 2022] PUT /identity/v3/projects/be6357215c1646169a897b22529b055d/users/d74f77cb251e4207bdcaf9e6db5f7f31/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:59.027485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 899/1795] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:56:59.027599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.036836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.038416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.039930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.144190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] MFA Rules not processed for user `d74f77cb251e4207bdcaf9e6db5f7f31`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:59.148821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:59.261826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:59.310920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee752e-7ccf-4fda-a915-be58d5ab5f58 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:56:59.321362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:56:59.333923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:56:59.335436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.337509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.339349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] PATH_INFO: `/v3/projects/929ff9715d7745a68938b657e678ee37/users/89b21fcb7d0f4bcf992c8bf295773f0a/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.373753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=929ff9715d7745a68938b657e678ee37, user_id=89b21fcb7d0f4bcf992c8bf295773f0a, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:56:59.379787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dba309-e82a-4b22-974c-ffb1155f2d92 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:56:59.420189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 897/1796] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:56:59 2022] PUT /identity/v3/projects/929ff9715d7745a68938b657e678ee37/users/89b21fcb7d0f4bcf992c8bf295773f0a/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:56:59.422787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:56:59.423152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:56:59.427720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:56:59.527326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] MFA Rules not processed for user `89b21fcb7d0f4bcf992c8bf295773f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:56:59.528651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:59.647261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 900/1797] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 618 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:59.648381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.663164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:56:59.867458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 898/1798] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:56:59.867458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:56:59.908336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157f8da-2786-4c16-8b18-642c65d023d5 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.181405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0a6f2-5d39-47b3-8ac3-e1d775922e17 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.186837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.189077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.190164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.190450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.209517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.231154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.235132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.236163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.240113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.242273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.380552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 899/1799] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:56:59 2022] POST /identity/v3/projects => generated 365 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:00.384565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.391737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.398886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.441126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 901/1800] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:56:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 793 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:00.530723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:00.609667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.620598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286ef85-f537-477a-ab0d-eaf708bd0943 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.642833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28b599-7891-4487-b8d6-2d75bdb1b4ff service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.654300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.658667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.659015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.665079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.666095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.667102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.667963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.668076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.668313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.669894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] PATH_INFO: `/v3/users/e23bb718ad6745b0b822fd58613d5430` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.678521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23bb718ad6745b0b822fd58613d5430)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.679188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.724710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 900/1801] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:00 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:00.726491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.729734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:00.775085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:00.796896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035893d-88c9-4722-8bf0-315c3b1a60c9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:00.804612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:00.805935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:00.806799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:00.807307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] PATH_INFO: `/v3/projects/f41b10ff2d8e49ceae763c34c0a29ae9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:00.809889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41b10ff2d8e49ceae763c34c0a29ae9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:00.815554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:00.835993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23bb718ad6745b0b822fd58613d5430'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.867152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23bb718ad6745b0b822fd58613d5430'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.868221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23bb718ad6745b0b822fd58613d5430'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.897203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23bb718ad6745b0b822fd58613d5430'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.920704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23bb718ad6745b0b822fd58613d5430'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.963094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 902/1802] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:00 2022] DELETE /identity/v3/users/e23bb718ad6745b0b822fd58613d5430 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:00.966591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:00.969104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41b10ff2d8e49ceae763c34c0a29ae9', 'request_id': 'req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:00.972868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:00.987227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41b10ff2d8e49ceae763c34c0a29ae9', 'request_id': 'req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.014424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41b10ff2d8e49ceae763c34c0a29ae9', 'request_id': 'req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.057853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] The token cache is being invalidate because project f41b10ff2d8e49ceae763c34c0a29ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:01.058152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.061539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d4bb297e-fad3-4de6-ae1f-3881f84b8c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.073589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 901/1803] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:00 2022] DELETE /identity/v3/projects/f41b10ff2d8e49ceae763c34c0a29ae9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:01.073589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.073589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.073589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.101609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:01.103317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.186796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8511-cd2b-4302-801d-9fadd66f0c7f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.196344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.198380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.198740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.199188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.200248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.205299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.243789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 903/1804] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:01.246609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.250619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.280603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173b8eb-3029-4a93-8d62-dccec8a0b01d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.302099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.304660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.307331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.307331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] PATH_INFO: `/v3/users/2a17995bfeab40a4a7c2e4da83e094cb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.312677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a17995bfeab40a4a7c2e4da83e094cb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.316408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.356354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 902/1805] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:01.357377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.360695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.424548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf67a1-84f7-4f53-a302-065f4b15b473 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.455640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.461043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.461333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.461804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] PATH_INFO: `/v3/projects/7a0c7b66807e4d52872e8d8a9b250336/users/8fc378b8b9544682a88d36c6af963866/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.514695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a17995bfeab40a4a7c2e4da83e094cb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.530463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a17995bfeab40a4a7c2e4da83e094cb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.531285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a17995bfeab40a4a7c2e4da83e094cb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.541507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a17995bfeab40a4a7c2e4da83e094cb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.549956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0c7b66807e4d52872e8d8a9b250336, user_id=8fc378b8b9544682a88d36c6af963866, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:01.565854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a17995bfeab40a4a7c2e4da83e094cb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:01.567750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed5e30-f25f-4640-9abe-4b26a23c6c14 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:01.614696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 904/1806] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:01 2022] DELETE /identity/v3/users/2a17995bfeab40a4a7c2e4da83e094cb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:01.617533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 903/1807] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:01 2022] PUT /identity/v3/projects/7a0c7b66807e4d52872e8d8a9b250336/users/8fc378b8b9544682a88d36c6af963866/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:01.618596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:01.620624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.623228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.639097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.643783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.693903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:01.709593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:01.716359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.874796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243c012-3e54-4c12-8dc6-c476e124f44d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:01.884326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:01.885134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:01.885458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:01.885743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:01.897956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:01.962626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:02.021522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 904/1808] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jul 14 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:02.022780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:02.025156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:02.140701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:02.142510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:02.164004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 905/1809] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:02.166802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca3315-6364-47e8-9676-2eb2956a79a4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:02.178208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:02.179445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:02.179976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:02.180473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:02.181439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:02.183852 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746a4c5-381d-4eb7-8403-0aee89a07cc6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:02.205745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 905/1810] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:02.211006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:02.228639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f30f6-daee-4f49-8dc9-f14909c4b61a service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:02.240921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:02.244348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:02.244636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:02.244922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] PATH_INFO: `/v3/projects/7a0c7b66807e4d52872e8d8a9b250336/users/8fc378b8b9544682a88d36c6af963866/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:02.250208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0c7b66807e4d52872e8d8a9b250336, user_id=8fc378b8b9544682a88d36c6af963866, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:02.258756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:02.320524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 906/1811] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:02 2022] PUT /identity/v3/projects/7a0c7b66807e4d52872e8d8a9b250336/users/8fc378b8b9544682a88d36c6af963866/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:02.322546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:02.322989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:02.323574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:02.412082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] MFA Rules not processed for user `8fc378b8b9544682a88d36c6af963866`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:02.422413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:02.535472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:02.691024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 906/1812] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:02.692859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:02.695228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:02.761263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce614c51-25a8-4132-8a50-e851bf58ef6d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:02.777683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:02.778856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:02.779153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:02.792915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:02.813791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:02.818796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:02.861870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 907/1813] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:02.862576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:02.866996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fc0fe-5fd7-4e6f-b89a-4bb7ac8061bb None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:02.870994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:02.893607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:02.894330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:02.894609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:02.894880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:02.925726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:02.925726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:02.929480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 907/1814] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:03.040249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af4da7-3933-4977-8330-38b1ef31ffac service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:03.046126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:03.046695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:03.047033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:03.047327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:03.068114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:03.080334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:03.094250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:03.145226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 908/1815] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:03.282470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:03.284651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:03.323445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f774-04ab-443a-a4c0-cbf85c7875e9 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:03.346113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:03.346113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:03.346113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:03.346113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:03.355657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:03.372174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:03.546184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:03.549068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:03.569930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 908/1816] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:05.926733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:05.929092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:06.242696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7a232-7f88-4796-9bf7-7bfc07262485 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:06.253283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:06.254933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.255571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.256259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] PATH_INFO: `/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.258472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ade901ce67742d5ae6d6b564d1ee1bd)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:06.262384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:06.352874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ade901ce67742d5ae6d6b564d1ee1bd', 'request_id': 'req-1a2f2d5c-7770-4a7f-a023-e296461c25bd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.370074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ade901ce67742d5ae6d6b564d1ee1bd', 'request_id': 'req-1a2f2d5c-7770-4a7f-a023-e296461c25bd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.396188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ade901ce67742d5ae6d6b564d1ee1bd', 'request_id': 'req-1a2f2d5c-7770-4a7f-a023-e296461c25bd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.396548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:06.398516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:06.416304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] The token cache is being invalidate because project 2ade901ce67742d5ae6d6b564d1ee1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:06.417736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.418984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1a2f2d5c-7770-4a7f-a023-e296461c25bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.429798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 909/1817] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:05 2022] DELETE /identity/v3/projects/2ade901ce67742d5ae6d6b564d1ee1bd => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:06.441561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483330e-b3dc-4b1a-92be-2b27bf8a9c5e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.441938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483330e-b3dc-4b1a-92be-2b27bf8a9c5e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.442290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483330e-b3dc-4b1a-92be-2b27bf8a9c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.447717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0d374-2614-4b8d-86b2-23062157a7fe service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:06.533633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c483330e-b3dc-4b1a-92be-2b27bf8a9c5e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:06.534903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c483330e-b3dc-4b1a-92be-2b27bf8a9c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:06.762563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:06.766936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.767596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.768392 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] PATH_INFO: `/v3/projects/528b258b867940339e81c2470f1b72f2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.775434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528b258b867940339e81c2470f1b72f2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:06.786138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:06.846366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 910/1818] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:06.858132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:06.858650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:06.859279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:06.920955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:06.925971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:06.945263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528b258b867940339e81c2470f1b72f2', 'request_id': 'req-424fa6b6-8be3-4503-85fd-0e0c820cb280'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.964174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528b258b867940339e81c2470f1b72f2', 'request_id': 'req-424fa6b6-8be3-4503-85fd-0e0c820cb280'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:06.978321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528b258b867940339e81c2470f1b72f2', 'request_id': 'req-424fa6b6-8be3-4503-85fd-0e0c820cb280'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.012126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:07.014358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] The token cache is being invalidate because project 528b258b867940339e81c2470f1b72f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:07.014664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.015702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.022555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 909/1819] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:06 2022] DELETE /identity/v3/projects/528b258b867940339e81c2470f1b72f2 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:07.023524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:07.028633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:07.171257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:07.202546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 911/1820] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:07.204278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:07.247154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fa6b6-8be3-4503-85fd-0e0c820cb280 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:07.262997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69335cef-0dcc-4055-a11f-67a043762153 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:07.266146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:07.268723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.269425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.270777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] PATH_INFO: `/v3/users/d74f77cb251e4207bdcaf9e6db5f7f31` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.274299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74f77cb251e4207bdcaf9e6db5f7f31)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:07.279744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:07.294933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:07.295751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.296074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.300024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.303685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:07.304167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:07.311762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:07.349967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 912/1821] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:07.354614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:07.381006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c2e05-dd43-49ad-9a32-edf15f06a60e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:07.402004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:07.402703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.405770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.406168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.407003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:07.409327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:07.418310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74f77cb251e4207bdcaf9e6db5f7f31'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.455553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74f77cb251e4207bdcaf9e6db5f7f31'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.457575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74f77cb251e4207bdcaf9e6db5f7f31'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.483413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 913/1822] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:07 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:07.485452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:07.487963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74f77cb251e4207bdcaf9e6db5f7f31'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.500249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-aec482c6-ba72-46ab-bea8-22a643f369f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74f77cb251e4207bdcaf9e6db5f7f31'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:07.511175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 910/1823] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:07 2022] DELETE /identity/v3/users/d74f77cb251e4207bdcaf9e6db5f7f31 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:07.638153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ad1-217d-44a0-a563-a9c5c00cd2ae admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:07.832835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.833119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.833393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.834812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 911/1824] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:07.838455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:07.840584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:07.840866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:07.841148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:07.841895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:07.864862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:07.905035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 914/1825] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:07 2022] POST /identity/v3/users => generated 287 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:07.928749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:07.931597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:07.992189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f1f2d-ddf3-4330-be58-e986f02cb26e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:08.007374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:08.008452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.008802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.010667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.011483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:08.018100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:08.055707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 912/1826] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:08.056526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.095646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfecf4-f1fa-4724-b4f7-9b6fa763c57f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:08.109108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:08.109817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.110064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.110339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.122455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:08.137726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaecace-e235-4ac3-b9e5-3018f13f0c69 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:08.200795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.213168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 915/1827] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:08.224763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f630d3c-7520-4cef-8535-9faf178513d3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:08.250484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:08.252146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.252670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.253310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] PATH_INFO: `/v3/projects/ebcec72e0b7645acb6d86fbaf1d974ef/users/0f6923b6655e451fa06edeccd25979c6/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.256618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467ae49-ae51-4c07-a3e6-9a4d19608b78 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.256895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467ae49-ae51-4c07-a3e6-9a4d19608b78 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.257172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467ae49-ae51-4c07-a3e6-9a4d19608b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.299523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcec72e0b7645acb6d86fbaf1d974ef, user_id=0f6923b6655e451fa06edeccd25979c6, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:08.308277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2ab1c-38d3-4dcb-9b76-1dc2d414d4f5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:08.314090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a467ae49-ae51-4c07-a3e6-9a4d19608b78 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:08.318308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a467ae49-ae51-4c07-a3e6-9a4d19608b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:08.343533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 913/1828] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:08 2022] PUT /identity/v3/projects/ebcec72e0b7645acb6d86fbaf1d974ef/users/0f6923b6655e451fa06edeccd25979c6/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:08.349573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.349990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.350370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.382449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:08.392339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:08.394297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:08.682797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 916/1829] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:57:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:08.685993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 914/1830] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:08.687498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:08.693539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:08.693880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:08.695781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 917/1831] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:57:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:08.964549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:08.966431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.025441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:09.143144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e592-0412-4d16-a085-01ac44a6d7db None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.155243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.158604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.186336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.187195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.187618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.192840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.204716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ce838-dbec-48df-9eee-118b39e90bc9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.213914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.215375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.215810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.216350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] PATH_INFO: `/v3/users/89b21fcb7d0f4bcf992c8bf295773f0a` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.218440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b21fcb7d0f4bcf992c8bf295773f0a)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.220827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.225997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1ccf7-4fdc-4940-af78-433c752f880e service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.227442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.263665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 915/1832] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:09.268472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.271331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.271764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.316557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:09.316557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.323921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 916/1833] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:57:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:09.331350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b21fcb7d0f4bcf992c8bf295773f0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:09.346946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.354046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.361160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b21fcb7d0f4bcf992c8bf295773f0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:09.364141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b21fcb7d0f4bcf992c8bf295773f0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:09.383585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b21fcb7d0f4bcf992c8bf295773f0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:09.403155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0443-0d6b-489a-b7b9-3df67cbd9981 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.406436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-24e6b4e0-e2d4-4ed3-a035-2d131c3bca6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b21fcb7d0f4bcf992c8bf295773f0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:09.423092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.426955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.430451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 918/1834] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:09 2022] DELETE /identity/v3/users/89b21fcb7d0f4bcf992c8bf295773f0a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:09.430601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.430601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.433262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.433868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.434807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.435317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.463363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.466792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.487124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:09.491224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.495204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 917/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:09.658100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 919/1836] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:09.683944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:09.687523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:09.737723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075de39-db78-43dd-b780-3f0b78e257a0 service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:09.841089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:09.842134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:09.842134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:09.842791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:09.857068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:09.859700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:09.875503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 918/1837] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:10.874281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:10.876210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:10.911573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:11.028890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08588c-036b-47d0-af17-30d9ef91278a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:11.036202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:11.037208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:11.037998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:11.038282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:11.041218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:11.061877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:11.065234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f29ada-7842-445f-adf3-b0d919887fd9 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:11.082642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 920/1838] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:11.453159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:11.456104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:11.522425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83da26-a324-4035-afd3-063cc2ab8757 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:11.559014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:11.559932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:11.562607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:11.562952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] PATH_INFO: `/v3/projects/be6357215c1646169a897b22529b055d` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:11.566938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6357215c1646169a897b22529b055d)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:11.571201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:11.737086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6357215c1646169a897b22529b055d', 'request_id': 'req-19a2dcdf-5f22-4211-beb8-dd60827e370e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:11.752877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6357215c1646169a897b22529b055d', 'request_id': 'req-19a2dcdf-5f22-4211-beb8-dd60827e370e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:11.759355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6357215c1646169a897b22529b055d', 'request_id': 'req-19a2dcdf-5f22-4211-beb8-dd60827e370e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:11.775625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] The token cache is being invalidate because project be6357215c1646169a897b22529b055d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:11.775977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:11.776747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-19a2dcdf-5f22-4211-beb8-dd60827e370e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:11.778512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 919/1839] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:11 2022] DELETE /identity/v3/projects/be6357215c1646169a897b22529b055d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:11.784413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:11.784707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:11.784986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:11.809326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:11.810914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:11.914914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:11.916078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:11.916872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:11.919293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 921/1840] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:11.980447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:11.981791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:11.996015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:11.996233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 920/1841] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:12.008342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5eac5-9e9a-4604-9a55-e4139eafa9bc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.015589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.016349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.016949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.017228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.019212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:12.019547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.021279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.040708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 922/1842] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:12.045628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.058185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65abc-a167-4a4b-8d99-693295a11511 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.072328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.073124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.074554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.076420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.077314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.082185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.138255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 921/1843] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:12 2022] POST /identity/v3/projects => generated 399 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:12.139944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.160812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af8bc0-6d18-43c3-b85f-8bbbf2dcd0c6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.333305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.334301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.336044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.337038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.337922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.343959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.379759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.382713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:12.398136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:12.399619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 923/1844] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:12 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:12.401710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.414620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a0c1a-9786-4aae-b3d3-6083784b4950 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.426096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9576f23-e552-4281-9fb7-4a1424f8f487 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.427402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.428331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.428614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.428901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] PATH_INFO: `/v3/projects/929ff9715d7745a68938b657e678ee37` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.431500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=929ff9715d7745a68938b657e678ee37)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.434379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.442952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.443701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.446697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.447048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.447844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.451221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.477594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 924/1845] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:12.483676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.507022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55400476-e436-4dcd-b5a3-5f9cd52f464a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.521384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.522702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.524363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.524734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.526565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.532329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.545972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929ff9715d7745a68938b657e678ee37', 'request_id': 'req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.549291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:12.556199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929ff9715d7745a68938b657e678ee37', 'request_id': 'req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.563836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929ff9715d7745a68938b657e678ee37', 'request_id': 'req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.568474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 925/1846] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:12.569604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.582539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] The token cache is being invalidate because project 929ff9715d7745a68938b657e678ee37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:12.582873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.584284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.590402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 922/1847] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:12 2022] DELETE /identity/v3/projects/929ff9715d7745a68938b657e678ee37 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:12.590814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:12.593052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:12.604002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceb01c-ee2d-4e56-9ec9-e4c4fc8d41d7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.732095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4ff9c-4b83-4bd0-9791-c2bb9141f213 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:12.749298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.749298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.750797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.751101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] PATH_INFO: `/v3/users/8fc378b8b9544682a88d36c6af963866` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.752636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc378b8b9544682a88d36c6af963866)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.756981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:12.757241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.758333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.760007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.760718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] PATH_INFO: `/v3/projects/913a03148d054d0c8ee2238248c7debb/users/13566d5108974ed093fd9a16c12bf239/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.786649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913a03148d054d0c8ee2238248c7debb, user_id=13566d5108974ed093fd9a16c12bf239, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:12.789683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf5acd-d94b-420e-9100-624d340e2ad0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:12.808688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 926/1848] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:12 2022] PUT /identity/v3/projects/913a03148d054d0c8ee2238248c7debb/users/13566d5108974ed093fd9a16c12bf239/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:12.817559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.818605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.819170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.856966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] MFA Rules not processed for user `0f6923b6655e451fa06edeccd25979c6`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:12.858388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:12.860723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc378b8b9544682a88d36c6af963866'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.876057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc378b8b9544682a88d36c6af963866'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.877040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc378b8b9544682a88d36c6af963866'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.890523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc378b8b9544682a88d36c6af963866'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.905692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-dbc0897f-348c-4059-b773-ae07b26629fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc378b8b9544682a88d36c6af963866'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:12.927405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 923/1849] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:12 2022] DELETE /identity/v3/users/8fc378b8b9544682a88d36c6af963866 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:12.928702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:12.928991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:12.929279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:12.953454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:12.999698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] MFA Rules not processed for user `13566d5108974ed093fd9a16c12bf239`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:13.001332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.027563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 927/1850] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:13.028476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.030045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.130349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5a5d7-7730-44d9-9999-423d1d9450e5 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.138450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 924/1851] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:13.143835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.158478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.162052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.256914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.257593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.258654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.258945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.259654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.262455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.311503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf3484-64ec-45f6-b781-c1dc94c12f4f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.320849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.321675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.322012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 928/1852] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/projects => generated 369 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:13.322127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.322365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.323429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.326240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.327988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.365479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 925/1853] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/projects => generated 401 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:13.366644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.377469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.421256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0181c-eec4-4306-b3b2-fe72cc4b3ee4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.458354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634b3f-3d96-406d-8c08-3963b5167c3d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.463076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.463968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.464246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.464516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] PATH_INFO: `/v3/projects/7a0c7b66807e4d52872e8d8a9b250336` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.465692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0c7b66807e4d52872e8d8a9b250336)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.467975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.483941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.484558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.484863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.485132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.485820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.487721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.491556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.509835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 929/1854] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:13.510635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.517819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0c7b66807e4d52872e8d8a9b250336', 'request_id': 'req-d691ede2-986b-4b71-b92a-149018f1e388'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:13.520697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae622fa4-22f0-4304-bb14-648b7a720e22 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.524204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0c7b66807e4d52872e8d8a9b250336', 'request_id': 'req-d691ede2-986b-4b71-b92a-149018f1e388'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:13.529922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0c7b66807e4d52872e8d8a9b250336', 'request_id': 'req-d691ede2-986b-4b71-b92a-149018f1e388'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:13.542657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] The token cache is being invalidate because project 7a0c7b66807e4d52872e8d8a9b250336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:13.542943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:13.543560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:13.545201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 926/1855] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:13 2022] DELETE /identity/v3/projects/7a0c7b66807e4d52872e8d8a9b250336 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:13.546226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.547574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.561225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.580040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.580723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.581034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.581316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.582046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.584209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f6afd-ef40-42ce-9c43-4dc32694ee2a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.608515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 930/1856] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:13.611094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.611885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.612402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.618027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691ede2-986b-4b71-b92a-149018f1e388 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.624470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.625407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.625688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.625948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.626689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.628828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.633505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:13.635601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.639535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 927/1857] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:13.640652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.644081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.653647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 931/1858] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:13.653647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.657711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2153e66-ce87-4a19-8106-967545ccec02 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.663327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.664122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.664418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.664666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.665397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.665617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbd5c3-6814-4d20-9df6-b8952ff455fc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.667083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f93157-16f1-4ce0-a95e-9b2dfd4390dc admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.672347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.673902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.674191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.674701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] PATH_INFO: `/v3/projects/689823c80b954defa28f85c6ef1333a1/users/2d338ad4ef764f23818378a28cec7467/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.680740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 928/1859] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:13.680740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.680740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.680740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.696929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689823c80b954defa28f85c6ef1333a1, user_id=2d338ad4ef764f23818378a28cec7467, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.699191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:13.699630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.700413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:13.709115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 929/1860] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:13.709952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.716271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 932/1861] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:13 2022] PUT /identity/v3/projects/689823c80b954defa28f85c6ef1333a1/users/2d338ad4ef764f23818378a28cec7467/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:13.717086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.720116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6532169-e35e-491f-a4bc-31e62012476a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.777222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.778021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.778297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.778562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] PATH_INFO: `/v3/projects/703fa403c6cb45daa917fec5bc5c714f/users/045f153600c243d09d7e55039f9f6833/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.781668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b922-5be4-41fd-b54e-54c4ad970702 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.786185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.786844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.787133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.787421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.788169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:13.788477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.790101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.792874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703fa403c6cb45daa917fec5bc5c714f, user_id=045f153600c243d09d7e55039f9f6833, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.795672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.802096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 933/1862] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:13.804026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.808947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 930/1863] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:13 2022] PUT /identity/v3/projects/703fa403c6cb45daa917fec5bc5c714f/users/045f153600c243d09d7e55039f9f6833/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:13.809721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.817524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b452ce-419a-4459-89fb-1d90691b8d5a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.821468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:13.876567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d2617-978b-4975-9312-e554099effd2 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.881329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.881990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.882269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.882530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.883223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.885163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.890400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:13.891075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:13.891354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:13.891972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:13.892735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:13.894903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:13.908503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 934/1864] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:13.909563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.917288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 931/1865] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:13 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:13.917288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:13.925656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fbbd9-23e9-4c3e-ac61-982a23196999 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:13.973547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.016428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eec96-fcdd-4b70-9eb3-cc2c78da807c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.018802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.019560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.019950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.020297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.021121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.022022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.022887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.023702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.024035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.024583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] PATH_INFO: `/v3/projects/689823c80b954defa28f85c6ef1333a1/users/2d338ad4ef764f23818378a28cec7467/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.027374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=689823c80b954defa28f85c6ef1333a1, user_id=2d338ad4ef764f23818378a28cec7467, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.038145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 935/1866] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:14.039148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.045591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.051858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0262b-cd11-47c9-b06c-aaeb0ad1ae7e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.059625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.059625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.059625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.059625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.060822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.063112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.071446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 932/1867] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:13 2022] PUT /identity/v3/projects/689823c80b954defa28f85c6ef1333a1/users/2d338ad4ef764f23818378a28cec7467/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:14.072521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.094482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 936/1868] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:14 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:14.099024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.116902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826186b-8f95-4167-ac54-db8a88467726 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.193628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550e610-5522-4fa5-9e24-0f7eea86516d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.206652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.207554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.207825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.208267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] PATH_INFO: `/v3/projects/703fa403c6cb45daa917fec5bc5c714f/users/045f153600c243d09d7e55039f9f6833/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.212606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=703fa403c6cb45daa917fec5bc5c714f, user_id=045f153600c243d09d7e55039f9f6833, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.216282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959bfe1-9ff3-44fa-a686-c829b644f61d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.221391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.222149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.222687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.223043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.224374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.225913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.240661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 937/1869] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:14.243042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 933/1870] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:14 2022] PUT /identity/v3/projects/703fa403c6cb45daa917fec5bc5c714f/users/045f153600c243d09d7e55039f9f6833/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:14.249374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.254782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.255128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.255522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.270913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922f50-5893-4a15-a80b-47f4f81d0407 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.320625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] MFA Rules not processed for user `045f153600c243d09d7e55039f9f6833`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:14.321855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:14.336304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.350853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.352304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.352568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.352944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.354181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.355905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.370826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 938/1871] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:14.375004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.386523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef95b6-673d-4146-a57e-488013b07ea6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.392443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.393435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.394200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.394200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] PATH_INFO: `/v3/projects/a162e3d58afd4f25af8df32db2a45f3c/users/8267cce304f749c1999401fcdedeca67/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.409222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 934/1872] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:14.416533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a162e3d58afd4f25af8df32db2a45f3c, user_id=8267cce304f749c1999401fcdedeca67, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.419641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defe46d-377d-4251-8e5f-e5726a8c43e7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.421724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.425425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:14.451918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 939/1873] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:14 2022] PUT /identity/v3/projects/a162e3d58afd4f25af8df32db2a45f3c/users/8267cce304f749c1999401fcdedeca67/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:14.453651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.453946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.454228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.505099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] MFA Rules not processed for user `8267cce304f749c1999401fcdedeca67`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:14.506347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:14.522576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.553978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cae2b-3dc2-43ea-9927-2d928d8ce0ef None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.606251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.607353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.607353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.607538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.614067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 940/1874] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:14.619021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.675525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.677431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.688451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 935/1875] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:14.720054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e14dc1-660c-47ce-a5a9-2344a2e2389a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.733636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.735327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.735889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.737168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.738187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.740393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.777380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 941/1876] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:14 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:14.781801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.797116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc430a1-ac46-44f9-992e-a8ca5ae373e0 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.813067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:14.817060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.911587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899522-42e7-414e-adeb-7881ae8484d0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.911789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.912593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.913079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.913452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.914892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.918744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.921619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.922523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.922806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.923082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.925493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:14.948622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 936/1877] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:14 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:14.952287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:14.969532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568595-0a20-4c63-9247-a3e9f1c18ed0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:14.977280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:14.978105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:14.978482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:14.978887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:14.979714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:14.982303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:14.990348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.005730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 937/1878] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:15.011331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.027347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.028574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff4fe1-b2bf-49b0-9326-3aed4e7a2368 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.029216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.034788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.035444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.035782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.036097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.036901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.039170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8be9f-c083-4460-aed2-e20118572048 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.045206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 942/1879] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:15.053375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 938/1880] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:15.056648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.072772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672755fc-231e-420e-a515-2096c67773e1 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.079295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.080443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.080831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.081178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] PATH_INFO: `/v3/projects/43462268b5934771a466662635dbb308/users/945964765e9f483cb7e98e760fc16485/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.099409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43462268b5934771a466662635dbb308, user_id=945964765e9f483cb7e98e760fc16485, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.103672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.126151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 943/1881] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:15 2022] PUT /identity/v3/projects/43462268b5934771a466662635dbb308/users/945964765e9f483cb7e98e760fc16485/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:15.138855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.139452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.139965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.175323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] MFA Rules not processed for user `945964765e9f483cb7e98e760fc16485`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:15.177423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:15.265011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 939/1882] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:15.270041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.383726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e031-2316-460d-8834-b64539688e94 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.390433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.391448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.392445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.392821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.393847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.396470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339fa62-f14e-4093-8daa-da0b98096ef3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.434285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 944/1883] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:15 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:15.438787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.472262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.473348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.473747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.474817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5fde-ddb5-4d3a-9d5d-c357b4ebc55a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.510462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] MFA Rules not processed for user `2d338ad4ef764f23818378a28cec7467`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:15.511743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:15.622449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.623343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.625028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.625028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.626933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.631165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.636781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 945/1884] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:15.661465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.674310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 940/1885] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:15 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:15.679344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.694989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e5da-c73b-412a-b679-06d8bf48b4a4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.696475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.704963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.706078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.706374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.706906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.707827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.710311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.720389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:15.725676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 941/1886] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:15.730890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.748521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa570f-06ab-4f47-9f52-2eb12a7e8374 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.751073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d5ce8-583e-416a-a4c3-4bae1e6f4fc3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.752791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.753607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.754040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.754318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.756150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.756281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:15.757538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.757538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.757907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] PATH_INFO: `/v3/projects/c7b1db714188484da3ed786b4d7ed093/users/2cb5b571291b4fd792a0b12c91cbf95b/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.773666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b1db714188484da3ed786b4d7ed093, user_id=2cb5b571291b4fd792a0b12c91cbf95b, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.776472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.793369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 942/1887] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:15 2022] PUT /identity/v3/projects/c7b1db714188484da3ed786b4d7ed093/users/2cb5b571291b4fd792a0b12c91cbf95b/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:15.797416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.826117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.827719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.839740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 946/1888] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:15.868279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fee714-cfdf-430f-8be3-59b44d8672a4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.884140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.884903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.885314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.886320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.887213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.889352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280886b7-5609-4a2f-a0b5-543a679f8945 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.901891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 943/1889] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:15.905642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:15.919246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdb54b-e55e-456e-9937-f5b23f6f4335 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:15.925809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:15.926961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.927240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.927508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] PATH_INFO: `/v3/projects/c7b1db714188484da3ed786b4d7ed093/users/2cb5b571291b4fd792a0b12c91cbf95b/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.929548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b1db714188484da3ed786b4d7ed093, user_id=2cb5b571291b4fd792a0b12c91cbf95b, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:15.932329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:15.949085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 947/1890] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:15 2022] PUT /identity/v3/projects/c7b1db714188484da3ed786b4d7ed093/users/2cb5b571291b4fd792a0b12c91cbf95b/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:15.959250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:15.959544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:15.959825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:15.988273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] MFA Rules not processed for user `2cb5b571291b4fd792a0b12c91cbf95b`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:15.989641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:16.075638 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 944/1891] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:16.084979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:16.086882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:16.157394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b120ba-b882-4b83-acf9-a74f89b1837e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:16.161233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:16.161954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:16.162237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:16.162515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:16.177145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:16.179052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:16.193385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 948/1892] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:16.786762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:16.797275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94eea12-5faa-41ea-824f-bf8c764cdd48 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:16.802056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:16.802717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:16.803001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:16.803280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:16.851203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:16.865193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:16.866569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:16.876276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 945/1893] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:18.363562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:18.372306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac726a10-8408-4566-9a1b-715add102ebc service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:18.376493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:18.377123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:18.377397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:18.377645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:18.442395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:18.443835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b38d1-6fd0-4026-ba58-7cc5a46ff330 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:18.454683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 949/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:18 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:21.658532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:21.753420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb80e8-f620-4910-9e7c-e7284bc2f501 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:21.759925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:21.761113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.761421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.762323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] PATH_INFO: `/v3/users/13566d5108974ed093fd9a16c12bf239` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.763862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] RBAC: Authorizing `identity:delete_user(user_id=13566d5108974ed093fd9a16c12bf239)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:21.766090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:21.833746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13566d5108974ed093fd9a16c12bf239'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:21.850197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13566d5108974ed093fd9a16c12bf239'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:21.851095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13566d5108974ed093fd9a16c12bf239'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:21.858922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13566d5108974ed093fd9a16c12bf239'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:21.868761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13566d5108974ed093fd9a16c12bf239'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:21.877512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 946/1895] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:21 2022] DELETE /identity/v3/users/13566d5108974ed093fd9a16c12bf239 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:21.882092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:21.882596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:21.882687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:21.901003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:21.902214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:21.913594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.002336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 950/1896] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:22.017714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.019855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:22.132363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5352c-7940-4d08-b76e-90fc763d213f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.136420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.137076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.137359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.137651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.148386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.150146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489209e8-c0a4-4cc5-9f9f-eba1b798fe2c service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.161643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 947/1897] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:22.629567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.631085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:22.645981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b0be-1cab-4f9e-aa65-8f834c7a85fb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.652021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.653045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.653600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.654073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] PATH_INFO: `/v3/projects/913a03148d054d0c8ee2238248c7debb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.655726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] RBAC: Authorizing `identity:delete_project(project_id=913a03148d054d0c8ee2238248c7debb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:22.657760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:22.695779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180343c-e7ea-4e6d-9aed-f859c8acb44f None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.696348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180343c-e7ea-4e6d-9aed-f859c8acb44f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.696458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180343c-e7ea-4e6d-9aed-f859c8acb44f None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.697983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 948/1898] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:22.738293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913a03148d054d0c8ee2238248c7debb', 'request_id': 'req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:22.742797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.743267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.745110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.786554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913a03148d054d0c8ee2238248c7debb', 'request_id': 'req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:22.793222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:22.794605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:22.801662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913a03148d054d0c8ee2238248c7debb', 'request_id': 'req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:22.803129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:22.829470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] The token cache is being invalidate because project 913a03148d054d0c8ee2238248c7debb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:22.829867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:22.830637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:22.839464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 951/1899] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:22 2022] DELETE /identity/v3/projects/913a03148d054d0c8ee2238248c7debb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:22.843441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:22.845357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:22.969802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 949/1900] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:57:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:22.988429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ebcb-4bc1-48ed-9580-e34aaa9eb54c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:22.995151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:22.996336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:22.996757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:22.997170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] PATH_INFO: `/v3/users/045f153600c243d09d7e55039f9f6833` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:22.998514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=045f153600c243d09d7e55039f9f6833)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.001388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.057621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.060309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:23.081319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e520f-1853-4dab-8199-b741cc8c03bd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.086792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:23.095935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.097478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.097983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.098434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.101545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045f153600c243d09d7e55039f9f6833'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.112559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.114598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.120154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045f153600c243d09d7e55039f9f6833'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.121013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045f153600c243d09d7e55039f9f6833'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.128435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 950/1901] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:23.129463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045f153600c243d09d7e55039f9f6833'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.167633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ad68a438-1d2f-42b3-b89e-8897f7240db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045f153600c243d09d7e55039f9f6833'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.185097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 952/1902] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:22 2022] DELETE /identity/v3/users/045f153600c243d09d7e55039f9f6833 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:23.670694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:23.696111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6192bd-9f15-4b20-80e5-960025e1846c service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:23.816193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:23.817432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.817940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.818266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] PATH_INFO: `/v3/projects/703fa403c6cb45daa917fec5bc5c714f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.820873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703fa403c6cb45daa917fec5bc5c714f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:23.823539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:23.901066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703fa403c6cb45daa917fec5bc5c714f', 'request_id': 'req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.908018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703fa403c6cb45daa917fec5bc5c714f', 'request_id': 'req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.922607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703fa403c6cb45daa917fec5bc5c714f', 'request_id': 'req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.942277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] The token cache is being invalidate because project 703fa403c6cb45daa917fec5bc5c714f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:23.942645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.943453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9fdbf8fc-a2cd-480b-92f8-66ee6e8035a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:23.945159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 951/1903] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:23 2022] DELETE /identity/v3/projects/703fa403c6cb45daa917fec5bc5c714f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:23.950137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:23.950522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:23.950870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:23.974481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:23.975936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:24.038327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 953/1904] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:24.046538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245454-d249-4669-bb4d-29391bdf14db None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.046848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245454-d249-4669-bb4d-29391bdf14db None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.047114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245454-d249-4669-bb4d-29391bdf14db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.065404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-99245454-d249-4669-bb4d-29391bdf14db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:24.066613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-99245454-d249-4669-bb4d-29391bdf14db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:24.074805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 952/1905] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:24.079409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.089750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c59ad-3259-4013-8e83-da1fa4d99502 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.094935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.095598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.095905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.096188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.097007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:24.097319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.098927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.113552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 954/1906] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:24.118099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99245454-d249-4669-bb4d-29391bdf14db None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.128448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99245454-d249-4669-bb4d-29391bdf14db None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.133685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.134351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.134684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.134965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.135666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.137788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.147795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:24.172250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 953/1907] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:24.173497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.184139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d58a7-3440-491d-b014-6683a67b8cca admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.248442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.249164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.250644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.250938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.251650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.253633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.288897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 955/1908] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:24.297120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.311320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788f551-98f1-4b5b-84c7-e7eaecbd8300 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.317001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.317673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.317955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.318226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.318962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.320803 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.336003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 954/1909] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:24.337701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.347745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a00c7-7673-4f18-a5bd-d685ad550639 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.353137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.354358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.354850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.355409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.356245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.358048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5e8ec-b5b0-48da-a76b-68eefae2f356 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.371914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 956/1910] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:24.373654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.384477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30ff0-24e1-4f44-8501-f0df7bdde975 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.390482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.391413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.391701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.392323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] PATH_INFO: `/v3/projects/d3982aee6e4b451aaaf44e8d2c0800e4/users/32ea533b84da4275b2f96b3ff9aa2dec/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.411307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3982aee6e4b451aaaf44e8d2c0800e4, user_id=32ea533b84da4275b2f96b3ff9aa2dec, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.414515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.432667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 955/1911] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:24 2022] PUT /identity/v3/projects/d3982aee6e4b451aaaf44e8d2c0800e4/users/32ea533b84da4275b2f96b3ff9aa2dec/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:24.452465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.452801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.453010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.492411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] MFA Rules not processed for user `32ea533b84da4275b2f96b3ff9aa2dec`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:24.504895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:24.510570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.512372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:24.531636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:24.622193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 957/1912] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:24.627150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.646110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0f97-a9fa-472a-9347-783c6d3c231a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.650541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.651551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.652096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.652569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.654833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:24.706896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8104b-03b7-44bd-8a9c-dcaf87744566 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.712399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.713133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.713424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.714041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.714811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.718502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.735587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.737292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.754936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 956/1913] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:24.756760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 958/1914] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:24.760492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.779323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206ce00-40e4-40b6-ab75-b67602b9a6f3 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.854810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.855500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.855783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.856093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.857110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.859340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.874931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:24.899158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 957/1915] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:24 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:24.901754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.927602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13d13-a263-46d6-912d-4d0c1316a07c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.933569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.934250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.934535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.934813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.941743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:24.943465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:24.956435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 959/1916] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:24.960801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:24.973350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d2709-5b6e-4925-a259-2f56b4a7e634 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:24.978411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:24.979500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:24.979913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:24.980289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] PATH_INFO: `/v3/projects/5ee45f65aab34eb593b8a0717ef663b1/users/3900652419d3497fb3adb3ace0edd8bc/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:24.998964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee45f65aab34eb593b8a0717ef663b1, user_id=3900652419d3497fb3adb3ace0edd8bc, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.002251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.021855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 958/1917] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:24 2022] PUT /identity/v3/projects/5ee45f65aab34eb593b8a0717ef663b1/users/3900652419d3497fb3adb3ace0edd8bc/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:25.023661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.034495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d7ca-4063-468e-abeb-b878a6ce1072 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.105457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.106205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.106547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.106919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.107709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.111816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d164d-c69b-4d46-a069-5c2ebdb42cef admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.133450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 960/1918] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:25.134875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.151608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ade0-f1d6-410d-98f3-fe37e43c4c55 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.159633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.161101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.161574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.161939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] PATH_INFO: `/v3/projects/5ee45f65aab34eb593b8a0717ef663b1/users/3900652419d3497fb3adb3ace0edd8bc/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.164936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee45f65aab34eb593b8a0717ef663b1, user_id=3900652419d3497fb3adb3ace0edd8bc, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.168633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.181370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 959/1919] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:25 2022] PUT /identity/v3/projects/5ee45f65aab34eb593b8a0717ef663b1/users/3900652419d3497fb3adb3ace0edd8bc/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:25.194360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847975-3589-4317-bf36-62292a5723c6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.194802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847975-3589-4317-bf36-62292a5723c6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.195151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847975-3589-4317-bf36-62292a5723c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.225672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-03847975-3589-4317-bf36-62292a5723c6 None None] MFA Rules not processed for user `3900652419d3497fb3adb3ace0edd8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:25.226951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-03847975-3589-4317-bf36-62292a5723c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.236924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:25.411910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 961/1920] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:25.522947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.632776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9787-0d0e-420d-b4ff-8329ce7a425f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.638418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.639357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.639829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.640225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.651266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.653255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.678384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 960/1921] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:25.833697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03847975-3589-4317-bf36-62292a5723c6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:25.835233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-03847975-3589-4317-bf36-62292a5723c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:25.848926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03847975-3589-4317-bf36-62292a5723c6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:25.852649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:25.853314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:25.853597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:25.853872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:25.947440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:25.949021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f97fed-8ece-4ec1-8718-40600417ec90 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:25.959936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 962/1922] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:27.434494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:27.435904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:27.509577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71941d9-2b12-4272-a679-930977cecea5 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:27.514243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:27.515223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.515507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.515775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] PATH_INFO: `/v3/users/8267cce304f749c1999401fcdedeca67` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.517561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8267cce304f749c1999401fcdedeca67)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:27.519493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:27.593906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8267cce304f749c1999401fcdedeca67'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:27.611435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8267cce304f749c1999401fcdedeca67'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:27.614403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8267cce304f749c1999401fcdedeca67'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:27.616390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:27.628743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8267cce304f749c1999401fcdedeca67'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:27.641769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8267cce304f749c1999401fcdedeca67'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:27.648773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 961/1923] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:27 2022] DELETE /identity/v3/users/8267cce304f749c1999401fcdedeca67 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:27.654235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.655907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.656217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.675165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:27.676788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:27.745291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 963/1924] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:27.755054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:27.756561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:27.816475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f7196-6263-4d0f-a3eb-8e6485c206a5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:27.820175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:27.820846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:27.821126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:27.821399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:27.830391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:27.831817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:27.841127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 962/1925] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:28.468011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:28.469682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:28.482398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786a27c-e65f-415a-bc70-b301639e49e6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:28.492422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:28.493346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:28.493746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:28.493984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] PATH_INFO: `/v3/projects/a162e3d58afd4f25af8df32db2a45f3c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:28.495223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a162e3d58afd4f25af8df32db2a45f3c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:28.497230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:28.536796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:28.546231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a162e3d58afd4f25af8df32db2a45f3c', 'request_id': 'req-b9966335-3bb1-4478-a99e-bd19b1769fb2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.551803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a162e3d58afd4f25af8df32db2a45f3c', 'request_id': 'req-b9966335-3bb1-4478-a99e-bd19b1769fb2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.558103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a162e3d58afd4f25af8df32db2a45f3c', 'request_id': 'req-b9966335-3bb1-4478-a99e-bd19b1769fb2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.572169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] The token cache is being invalidate because project a162e3d58afd4f25af8df32db2a45f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:28.572508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.573150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.574807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 964/1926] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:28 2022] DELETE /identity/v3/projects/a162e3d58afd4f25af8df32db2a45f3c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:28.579435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:28.594368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c93af-b9d4-49a3-8996-c5fe2324f03c service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:28.659914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:28.660801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:28.661070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:28.661332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] PATH_INFO: `/v3/users/945964765e9f483cb7e98e760fc16485` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:28.662558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=945964765e9f483cb7e98e760fc16485)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:28.664520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:28.711738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945964765e9f483cb7e98e760fc16485'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.723861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945964765e9f483cb7e98e760fc16485'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.724564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945964765e9f483cb7e98e760fc16485'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.732479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945964765e9f483cb7e98e760fc16485'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.740383 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945964765e9f483cb7e98e760fc16485'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:28.748429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 963/1927] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:28 2022] DELETE /identity/v3/users/945964765e9f483cb7e98e760fc16485 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:29.030358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.032418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:29.046504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9966335-3bb1-4478-a99e-bd19b1769fb2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.105504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.106289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.106579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.106861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] PATH_INFO: `/v3/projects/43462268b5934771a466662635dbb308` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.108121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43462268b5934771a466662635dbb308)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.110906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.161167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43462268b5934771a466662635dbb308', 'request_id': 'req-a703dcd2-c773-450c-b756-2d245fd39f4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.166597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43462268b5934771a466662635dbb308', 'request_id': 'req-a703dcd2-c773-450c-b756-2d245fd39f4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.172830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43462268b5934771a466662635dbb308', 'request_id': 'req-a703dcd2-c773-450c-b756-2d245fd39f4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.185813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] The token cache is being invalidate because project 43462268b5934771a466662635dbb308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:29.186103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.187201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.188927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 965/1928] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:29 2022] DELETE /identity/v3/projects/43462268b5934771a466662635dbb308 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:29.192967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.194360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:29.203929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.210203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5483b-e0a5-424a-9a53-4f9bc533c0fb admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.269097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.270005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.270293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.270561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] PATH_INFO: `/v3/users/2cb5b571291b4fd792a0b12c91cbf95b` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.271780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb5b571291b4fd792a0b12c91cbf95b)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.273954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.330668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb5b571291b4fd792a0b12c91cbf95b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.343021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb5b571291b4fd792a0b12c91cbf95b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.343866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb5b571291b4fd792a0b12c91cbf95b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.352925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb5b571291b4fd792a0b12c91cbf95b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.360990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e26f6100-83e6-4722-807f-e8f5423c1f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb5b571291b4fd792a0b12c91cbf95b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.367062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 964/1929] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:29 2022] DELETE /identity/v3/users/2cb5b571291b4fd792a0b12c91cbf95b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:29.722450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:29.723641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:29.750416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703dcd2-c773-450c-b756-2d245fd39f4c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:29.809195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:29.819608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:29.820958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.821285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.821563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] PATH_INFO: `/v3/projects/c7b1db714188484da3ed786b4d7ed093` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.822874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b1db714188484da3ed786b4d7ed093)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:29.825750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:29.881915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b1db714188484da3ed786b4d7ed093', 'request_id': 'req-5bc566bd-ce62-4234-85dc-d7b4a4a74596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.887358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b1db714188484da3ed786b4d7ed093', 'request_id': 'req-5bc566bd-ce62-4234-85dc-d7b4a4a74596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.894181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b1db714188484da3ed786b4d7ed093', 'request_id': 'req-5bc566bd-ce62-4234-85dc-d7b4a4a74596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.907795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] The token cache is being invalidate because project c7b1db714188484da3ed786b4d7ed093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:29.908315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.908967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5bc566bd-ce62-4234-85dc-d7b4a4a74596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:29.910556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 966/1930] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:29 2022] DELETE /identity/v3/projects/c7b1db714188484da3ed786b4d7ed093 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:29.916607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:29.917083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:29.917167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:29.936017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:29.937273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:30.001509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 965/1931] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:30.011744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.012154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.012521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.032049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:30.033290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:30.042531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 967/1932] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:30.047026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.057351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11efa0b-9de0-46f9-85ba-716997e28bf1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.062158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.062823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.063113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.063389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.064162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:30.064516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.066125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.078320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 966/1933] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:30.083239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.094116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf874ea-e266-4ffe-af79-898f63f65cb4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.100902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.102286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.102699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.102977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.103828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.106368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.137183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 968/1934] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:30.143652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.154479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a8e63-13ec-4619-a6fb-d656b39b0536 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.163434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:30.216537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.217192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.217635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.217766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.218476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.220420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.241918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 967/1935] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:30.246270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.257605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6183b7-bb06-45ad-839b-014c0db49526 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.263236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.263923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.264209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.264677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.265533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.267644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.279417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 969/1936] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:30.283925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.293932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b62c3-53a0-47b7-9c28-4adaeefc1cf2 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.300981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.301652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.301928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.302204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.302981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.305213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f28bfe-e3e4-4ba8-b46c-c4254e61a901 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.315985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 968/1937] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:30.320283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.330617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205baa-a309-4295-adeb-9c62bd8aa0e8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.335894 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.336835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.337114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.337386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] PATH_INFO: `/v3/projects/1dc25167f31f4ee181aa097d76da1368/users/dcf0941361ec486080b780a3a091bf20/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.352723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc25167f31f4ee181aa097d76da1368, user_id=dcf0941361ec486080b780a3a091bf20, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.355403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.369644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 970/1938] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:30 2022] PUT /identity/v3/projects/1dc25167f31f4ee181aa097d76da1368/users/dcf0941361ec486080b780a3a091bf20/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:30.382469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.382804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.383084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.411479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] MFA Rules not processed for user `dcf0941361ec486080b780a3a091bf20`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:30.412753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:30.491095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 969/1939] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:30.495658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.571083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2199fb-30b0-45dd-8ac5-228d337c0f93 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.576161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.576823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.577101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.577399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.578429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.580342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.609973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 971/1940] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:30.611186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.624369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ffe72-f230-4698-9c8b-473b0069dfa2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.706700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.707387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.707667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.707974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.708691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.710655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.740452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 970/1941] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:30.742367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.756622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c7ddb-a4ab-40a6-b0a7-6e0552852098 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.769257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.770512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.770512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.770830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.771586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.773558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.776732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:30.790743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 972/1942] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:30.797313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.810591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f91d40-9446-4cd0-b7df-4bbb59119315 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.819320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.820470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.822231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.822231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.822918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.824921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.828581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:30.846938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 971/1943] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:30.852791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.867658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be27f-4e65-4a5a-b290-eee6b6737bd6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.893224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.894139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.894423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.894698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] PATH_INFO: `/v3/projects/3be1a97079f6442490c3abf383b0efb3/users/3410ec138fd2415890fd319ca73487e8/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.901838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:30.903670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:30.931891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be1a97079f6442490c3abf383b0efb3, user_id=3410ec138fd2415890fd319ca73487e8, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.934405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861b1f-4172-423e-843d-22daf46c04a7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.943064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27600bf-23e6-4354-9bfb-d3a3c03e15e8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:30.955704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:30.957078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.957500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.957912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] PATH_INFO: `/v3/users/32ea533b84da4275b2f96b3ff9aa2dec` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:30.959282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 973/1944] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:30 2022] PUT /identity/v3/projects/3be1a97079f6442490c3abf383b0efb3/users/3410ec138fd2415890fd319ca73487e8/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:30.959798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ea533b84da4275b2f96b3ff9aa2dec)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:30.963011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:30.971071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:30.972251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:30.972540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.036369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] MFA Rules not processed for user `3410ec138fd2415890fd319ca73487e8`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:31.039502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.059046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ea533b84da4275b2f96b3ff9aa2dec'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:31.095450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ea533b84da4275b2f96b3ff9aa2dec'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:31.096690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ea533b84da4275b2f96b3ff9aa2dec'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:31.107367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ea533b84da4275b2f96b3ff9aa2dec'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:31.116802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-732f0b3d-ba2c-4d09-826a-5aca6daf8748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ea533b84da4275b2f96b3ff9aa2dec'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:31.128371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 972/1945] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:30 2022] DELETE /identity/v3/users/32ea533b84da4275b2f96b3ff9aa2dec => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:31.133057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.133474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.133866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.143260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 974/1946] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:31.151911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.153057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.154892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:31.156167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.234128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 973/1947] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:31.251517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfd23-1404-401b-89b2-6da4fb8821d9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.254761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.256667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.257073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.257749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.258022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.258303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.259143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.261163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.302939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 975/1948] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:31 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:31.311810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.343244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854df2ee-09ec-4b64-9db0-39b5403b1bef None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.390256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:31.395268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:31.419387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e1dc4-37c8-4aec-9978-96bd8c39f8bf admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.428015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.428952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.429272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.429562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.430753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.434575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.454834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.455689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.456052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.456333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.473846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 976/1949] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:31 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:31.478193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.480035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.481633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.490300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aa46-790e-4bba-9e67-338ae182dc25 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.495078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 974/1950] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:31.495541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.496260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.496548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.496847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.497613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.499327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.514444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 977/1951] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:31.516057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.535942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01f710-5214-4db5-a952-9e57a33917e0 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.541518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.542702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.543681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.543681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] PATH_INFO: `/v3/projects/a5244d8987ce42c990640b4bf903f0d8/users/26e98ccdc86147c49ade079321016e15/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.580907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5244d8987ce42c990640b4bf903f0d8, user_id=26e98ccdc86147c49ade079321016e15, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.583747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.609062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 975/1952] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:31 2022] PUT /identity/v3/projects/a5244d8987ce42c990640b4bf903f0d8/users/26e98ccdc86147c49ade079321016e15/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:31.610391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.626999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984634-1805-4a34-ba37-e4480dae41d4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.743085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.743782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.744236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.744652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.745407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.747580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46b250-5864-4c67-aaa6-79d652024e86 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.762333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 978/1953] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:31.767397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.784796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdba4ef-adfb-46f3-971e-fafc675c4a6c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:31.792269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:31.793420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.793682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.794054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] PATH_INFO: `/v3/projects/a5244d8987ce42c990640b4bf903f0d8/users/26e98ccdc86147c49ade079321016e15/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.797775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5244d8987ce42c990640b4bf903f0d8, user_id=26e98ccdc86147c49ade079321016e15, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:31.801257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:31.823155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 976/1954] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:31 2022] PUT /identity/v3/projects/a5244d8987ce42c990640b4bf903f0d8/users/26e98ccdc86147c49ade079321016e15/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:31.829802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:31.830068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:31.830347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:31.872009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] MFA Rules not processed for user `26e98ccdc86147c49ade079321016e15`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:31.873514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:31.975477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:31.977307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.075291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 979/1955] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:32.090893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:32.092790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.121057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353ba90-9f46-4b0a-9456-89474b363ddb admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.126489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.127325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.127596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.128013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] PATH_INFO: `/v3/projects/d3982aee6e4b451aaaf44e8d2c0800e4` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.129995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3982aee6e4b451aaaf44e8d2c0800e4)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.132525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.172456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:32.176459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8212c-6fc7-4e05-9426-6d55ff624f55 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.189074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3982aee6e4b451aaaf44e8d2c0800e4', 'request_id': 'req-bb43c414-2978-4fb4-b239-5c9156caee8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.196659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3982aee6e4b451aaaf44e8d2c0800e4', 'request_id': 'req-bb43c414-2978-4fb4-b239-5c9156caee8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.202629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3982aee6e4b451aaaf44e8d2c0800e4', 'request_id': 'req-bb43c414-2978-4fb4-b239-5c9156caee8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.219178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] The token cache is being invalidate because project d3982aee6e4b451aaaf44e8d2c0800e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:32.219496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.220187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.224749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 977/1956] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:31 2022] DELETE /identity/v3/projects/d3982aee6e4b451aaaf44e8d2c0800e4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:32.225781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:32.227103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.240102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.240801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.241089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.241370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.243317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.254345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:32.302527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43c414-2978-4fb4-b239-5c9156caee8d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.308237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.309098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.309416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.309696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] PATH_INFO: `/v3/users/3900652419d3497fb3adb3ace0edd8bc` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.310969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3900652419d3497fb3adb3ace0edd8bc)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.313160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.313535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.315278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.340069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 980/1957] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:32.371813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3900652419d3497fb3adb3ace0edd8bc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.377656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:32.381167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.387183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3900652419d3497fb3adb3ace0edd8bc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.387957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3900652419d3497fb3adb3ace0edd8bc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.397352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3900652419d3497fb3adb3ace0edd8bc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.408595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3900652419d3497fb3adb3ace0edd8bc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:32.409329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2eb609-263a-4ce6-bcc9-544a0f478340 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:32.417261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 978/1958] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:32 2022] DELETE /identity/v3/users/3900652419d3497fb3adb3ace0edd8bc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:32.418336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:32.419167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:32.419521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:32.419899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:32.442420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:32.540491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:32.542782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:32.556194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 981/1959] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:33.347705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:33.349759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.453071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c57a-c7d4-4d99-8d3f-c7d134b3d943 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:33.461091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:33.462105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.462501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.462793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] PATH_INFO: `/v3/projects/5ee45f65aab34eb593b8a0717ef663b1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.464926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee45f65aab34eb593b8a0717ef663b1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:33.466878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:33.567172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:33.569597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.593023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125f7d-3f70-4117-9f2f-9a2f5b6865a4 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:33.595362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:33.601805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee45f65aab34eb593b8a0717ef663b1', 'request_id': 'req-2549b7b8-8a93-4d32-ae22-35287bff4bdc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.614007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee45f65aab34eb593b8a0717ef663b1', 'request_id': 'req-2549b7b8-8a93-4d32-ae22-35287bff4bdc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.623199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee45f65aab34eb593b8a0717ef663b1', 'request_id': 'req-2549b7b8-8a93-4d32-ae22-35287bff4bdc'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.644790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] The token cache is being invalidate because project 5ee45f65aab34eb593b8a0717ef663b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:33.646111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.647017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2549b7b8-8a93-4d32-ae22-35287bff4bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.649313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 979/1960] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:33 2022] DELETE /identity/v3/projects/5ee45f65aab34eb593b8a0717ef663b1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:33.654973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2108-81e4-4d0e-88f3-672a8f1e27e7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.657040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2108-81e4-4d0e-88f3-672a8f1e27e7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.659674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2108-81e4-4d0e-88f3-672a8f1e27e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.688089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:33.697727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-45ee2108-81e4-4d0e-88f3-672a8f1e27e7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:33.699219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-45ee2108-81e4-4d0e-88f3-672a8f1e27e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.699385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:33.700381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.700610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.701510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] PATH_INFO: `/v3/users/5623e80e405b43c28404f32071b8e67e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.702385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5623e80e405b43c28404f32071b8e67e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:33.704870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:33.711041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 980/1961] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:33.719233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.719525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.719807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.739671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:33.741039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.751004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 981/1962] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:33.755247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:33.765726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a60a4-98df-4015-bf2e-9f21712218a5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:33.770808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:33.771478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.771814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.772127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.772995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:33.773307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:33.775126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:33.786394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5623e80e405b43c28404f32071b8e67e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.792106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 982/1963] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:33.796611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:33.807340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a1474-c4fb-4f88-a5c5-c433d4ed1287 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:33.810144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5623e80e405b43c28404f32071b8e67e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.811361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5623e80e405b43c28404f32071b8e67e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.813184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:33.813903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.814192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.814479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.815659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:33.817426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:33.823281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5623e80e405b43c28404f32071b8e67e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.837927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a243c79f-0dbe-4533-8433-94621156a014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5623e80e405b43c28404f32071b8e67e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:33.845452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 982/1964] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:33 2022] DELETE /identity/v3/users/5623e80e405b43c28404f32071b8e67e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:33.849834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:33.850148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:33.850967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:33.868043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 983/1965] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:33.871736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:33.880078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:33.880078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.882824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:33.989707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 983/1966] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:34.028967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.031489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:34.094089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94bbd-7587-46f5-a0f1-9447f871e666 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.106397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.107259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.109128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.109749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.111380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.113691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.157280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 984/1967] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:33 2022] POST /identity/v3/users => generated 282 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:34.158420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.178275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d67fd-9355-4e35-971c-fcc1d0b9afe6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.185701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c737a2-2bd1-4dfd-8635-3315e99dc03f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.187276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.188045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.188287 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.188952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.199836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.199836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.199836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.199836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.200561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.203413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.214592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.216295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.230975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 985/1968] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:34.235064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.237439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 984/1969] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:34.252281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9336-aac8-4b15-84e3-105b925b3131 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.260024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.260219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.260421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.260794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.261551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.263365 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1bfe1-7028-43de-add7-68be9af74a8d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.277176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 986/1970] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:34.282784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.297064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e1813-097e-4c58-96bf-a35148a2c608 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.314417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.315860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.316427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.317073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] PATH_INFO: `/v3/projects/f2da94a14ca94e13b067ae9fd79bbdac/users/d75355d82a89414da65c0a18ec637dd9/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.342757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:34.347731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2da94a14ca94e13b067ae9fd79bbdac, user_id=d75355d82a89414da65c0a18ec637dd9, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.350841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.377363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 985/1971] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:34 2022] PUT /identity/v3/projects/f2da94a14ca94e13b067ae9fd79bbdac/users/d75355d82a89414da65c0a18ec637dd9/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:34.389421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.390501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.392606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.469514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] MFA Rules not processed for user `d75355d82a89414da65c0a18ec637dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:34.471440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:34.524933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:34.573956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 987/1972] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:34.587614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.590681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:34.606916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.609819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:34.726101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8e833-485b-4e9a-9856-7b024a796dd1 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.735299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.735299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.735675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.736044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.766340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.766340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.791673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 986/1973] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:34.795151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fd1d-21d0-44ff-b29f-553a2402f5da None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:34.818687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:34.819405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:34.819736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:34.820044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] PATH_INFO: `/v3/projects/e0fcfa39b02e4e9ea773e8c23c3b9551` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:34.824616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fcfa39b02e4e9ea773e8c23c3b9551)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:34.829324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:34.921792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fcfa39b02e4e9ea773e8c23c3b9551', 'request_id': 'req-564c868d-49c9-441f-9304-227f18126dac'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:34.931066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fcfa39b02e4e9ea773e8c23c3b9551', 'request_id': 'req-564c868d-49c9-441f-9304-227f18126dac'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:34.938617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fcfa39b02e4e9ea773e8c23c3b9551', 'request_id': 'req-564c868d-49c9-441f-9304-227f18126dac'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:34.959063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] The token cache is being invalidate because project e0fcfa39b02e4e9ea773e8c23c3b9551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:34.959354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:34.960122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:34.964068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 988/1974] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:34 2022] DELETE /identity/v3/projects/e0fcfa39b02e4e9ea773e8c23c3b9551 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:34.966911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:34.985706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1fd6c-87ab-4ec5-a67e-13dfe82685be service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:35.120233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:35.121488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:35.121878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:35.122180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] PATH_INFO: `/v3/users/6b8e030eb42f43d8bcfbe0a1189120cc` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:35.123525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8e030eb42f43d8bcfbe0a1189120cc)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:35.127058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:35.205970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8e030eb42f43d8bcfbe0a1189120cc'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:35.238851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8e030eb42f43d8bcfbe0a1189120cc'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:35.240647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8e030eb42f43d8bcfbe0a1189120cc'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:35.270380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8e030eb42f43d8bcfbe0a1189120cc'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:35.297145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e459fd5-2e21-4aaf-a458-b3daa41ea66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8e030eb42f43d8bcfbe0a1189120cc'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:35.309536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 987/1975] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:34 2022] DELETE /identity/v3/users/6b8e030eb42f43d8bcfbe0a1189120cc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:36.014405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.016132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.037450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c868d-49c9-441f-9304-227f18126dac admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.120036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.120956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.121378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.121662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] PATH_INFO: `/v3/users/dcf0941361ec486080b780a3a091bf20` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.122910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf0941361ec486080b780a3a091bf20)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.125562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.143142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.192135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf0941361ec486080b780a3a091bf20'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:36.210240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf0941361ec486080b780a3a091bf20'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:36.211502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf0941361ec486080b780a3a091bf20'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:36.220344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf0941361ec486080b780a3a091bf20'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:36.230052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-09a89b3d-2c20-40f5-8a9a-58825ae1a1b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf0941361ec486080b780a3a091bf20'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:36.241583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 989/1976] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:36 2022] DELETE /identity/v3/users/dcf0941361ec486080b780a3a091bf20 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:36.246548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.247082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.247184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.269168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.270659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.280787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:36.372996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c9ce-16d0-4e36-89c6-988ef39e00e0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.373287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c9ce-16d0-4e36-89c6-988ef39e00e0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.373559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c9ce-16d0-4e36-89c6-988ef39e00e0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.377561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 990/1977] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:57:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:57:36.377737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 988/1978] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:36.384505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.384795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.385071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.402581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.404260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.440332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:36.445644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.518686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99a548-9c36-4fb5-b1ac-74c2ac053f08 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.527507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.528731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.529340 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.529823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.544190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.546309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.562210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 989/1979] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:36.591585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 991/1980] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:57:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:36.610733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.614621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.635026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311aa3-19f2-4ce7-ab89-9038e8d884d2 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.643368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.645622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.646381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.647696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.662571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.665556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:36.679486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 990/1981] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:36.924724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:36.926357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:36.949196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd7636-810e-492c-a0e4-3b8851c76a4d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:36.959255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:36.960971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:36.961976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:36.962420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] PATH_INFO: `/v3/projects/1dc25167f31f4ee181aa097d76da1368` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:36.964238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc25167f31f4ee181aa097d76da1368)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:36.967762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.023057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc25167f31f4ee181aa097d76da1368', 'request_id': 'req-6044169d-9bb4-46af-bd11-afddd8f8e758'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.030213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc25167f31f4ee181aa097d76da1368', 'request_id': 'req-6044169d-9bb4-46af-bd11-afddd8f8e758'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.038172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc25167f31f4ee181aa097d76da1368', 'request_id': 'req-6044169d-9bb4-46af-bd11-afddd8f8e758'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.071964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] The token cache is being invalidate because project 1dc25167f31f4ee181aa097d76da1368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:37.072369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.073258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.076664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 992/1982] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:36 2022] DELETE /identity/v3/projects/1dc25167f31f4ee181aa097d76da1368 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:37.080693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.093606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2b38-80e5-473e-a501-05cf3a43b589 service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.201222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.202355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.202645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.202949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] PATH_INFO: `/v3/users/3410ec138fd2415890fd319ca73487e8` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.204198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3410ec138fd2415890fd319ca73487e8)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.206353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.266933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.269604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3410ec138fd2415890fd319ca73487e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.282767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3410ec138fd2415890fd319ca73487e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.283451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3410ec138fd2415890fd319ca73487e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.291238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3410ec138fd2415890fd319ca73487e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.299751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3410ec138fd2415890fd319ca73487e8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.306622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 991/1983] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:37 2022] DELETE /identity/v3/users/3410ec138fd2415890fd319ca73487e8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:37.413057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.414541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:37.430597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044169d-9bb4-46af-bd11-afddd8f8e758 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.435868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.522667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.523552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.523837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.524142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] PATH_INFO: `/v3/projects/da6a71d345fc4d73800c548291810983` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.525776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6a71d345fc4d73800c548291810983)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.527624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.601115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6a71d345fc4d73800c548291810983', 'request_id': 'req-7c767bb0-74ff-4ba3-9231-3af5d25a6987'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.611295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.612861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:37.620838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6a71d345fc4d73800c548291810983', 'request_id': 'req-7c767bb0-74ff-4ba3-9231-3af5d25a6987'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.630663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23dc0-0cb0-4af3-97aa-a7b13e8bdc7e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.637795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6a71d345fc4d73800c548291810983', 'request_id': 'req-7c767bb0-74ff-4ba3-9231-3af5d25a6987'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.663727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] The token cache is being invalidate because project da6a71d345fc4d73800c548291810983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:37.664123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.664883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7c767bb0-74ff-4ba3-9231-3af5d25a6987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.666893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 993/1984] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:37 2022] DELETE /identity/v3/projects/da6a71d345fc4d73800c548291810983 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:37.672126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490e873-c6b6-4aea-b499-44ead890e232 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.672544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490e873-c6b6-4aea-b499-44ead890e232 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.672832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490e873-c6b6-4aea-b499-44ead890e232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.692172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3490e873-c6b6-4aea-b499-44ead890e232 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:37.693842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3490e873-c6b6-4aea-b499-44ead890e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:37.705302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.706190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.706475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.706755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] PATH_INFO: `/v3/projects/3be1a97079f6442490c3abf383b0efb3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.708024 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be1a97079f6442490c3abf383b0efb3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.710002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.763022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be1a97079f6442490c3abf383b0efb3', 'request_id': 'req-5ff84f3c-1b33-421f-a7d0-df34a691b28c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.768970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 994/1985] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:37.773359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be1a97079f6442490c3abf383b0efb3', 'request_id': 'req-5ff84f3c-1b33-421f-a7d0-df34a691b28c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.777617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.778154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.778653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.780346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be1a97079f6442490c3abf383b0efb3', 'request_id': 'req-5ff84f3c-1b33-421f-a7d0-df34a691b28c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.794458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] The token cache is being invalidate because project 3be1a97079f6442490c3abf383b0efb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:37.794660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.795330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:37.797347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 992/1986] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:37 2022] DELETE /identity/v3/projects/3be1a97079f6442490c3abf383b0efb3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:37.801228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.802066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:37.802638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:37.803617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:37.844648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.868003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:37.874703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 995/1987] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:37.879339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.882365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff84f3c-1b33-421f-a7d0-df34a691b28c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.887970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.888962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.889294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.889687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] PATH_INFO: `/v3/users/26e98ccdc86147c49ade079321016e15` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.891507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e98ccdc86147c49ade079321016e15)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.894424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.897514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f2a62-8447-4e77-ad88-43f3337f561f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.903324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.904245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.904591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.904935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.905865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:37.906177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.907909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.928970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 996/1988] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:37.928970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:37.946104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fe16a-ec27-4817-bf6a-3a5037aac9fd admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:37.953727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:37.954535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:37.954874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:37.955186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:37.955979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:37.959971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:37.975782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e98ccdc86147c49ade079321016e15'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.005143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 997/1989] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:37 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:38.006064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.006721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e98ccdc86147c49ade079321016e15'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.007558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e98ccdc86147c49ade079321016e15'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.018932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e98ccdc86147c49ade079321016e15'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.027536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e98ccdc86147c49ade079321016e15'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.054948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 993/1990] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:37 2022] DELETE /identity/v3/users/26e98ccdc86147c49ade079321016e15 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:38.092678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0a40a-e65d-451b-a672-43bd30ad57e2 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.209400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.247662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.248115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.248530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.250276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.255598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.314541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 998/1991] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:38 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:38.318593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.320016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.348398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90610f7d-c603-493e-89b0-2599321c2cdb admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.387843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba018-a002-429f-815d-98772c697180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.388740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba018-a002-429f-815d-98772c697180 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.389079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba018-a002-429f-815d-98772c697180 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.389388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba018-a002-429f-815d-98772c697180 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.392773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ba018-a002-429f-815d-98772c697180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.394611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ba018-a002-429f-815d-98772c697180 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.422299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.424090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.425620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 994/1992] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:38.428778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba018-a002-429f-815d-98772c697180 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.438569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47396b9f-3759-43d8-a4ed-839510cfd704 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.446945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.448279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.448279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.448512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] PATH_INFO: `/v3/users/0f6923b6655e451fa06edeccd25979c6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.450762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6923b6655e451fa06edeccd25979c6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.452903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.457724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba018-a002-429f-815d-98772c697180 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.478464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.484485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.484485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.484794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.488005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.491988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.509066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 995/1993] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:38.514995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.515670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.526913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6923b6655e451fa06edeccd25979c6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.539325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbd39d-c687-44f3-bf02-c18dabfad689 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.551832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6923b6655e451fa06edeccd25979c6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.552565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6923b6655e451fa06edeccd25979c6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.554159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.555082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.555362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.555635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] PATH_INFO: `/v3/projects/8fa410029b42486eb0b9228afc066ea1/users/090ccf06098744928d4d8e03e40e8acf/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.562649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6923b6655e451fa06edeccd25979c6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.574512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-56a98e99-6ad2-46a6-a968-b6c979ca6742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6923b6655e451fa06edeccd25979c6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.579236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa410029b42486eb0b9228afc066ea1, user_id=090ccf06098744928d4d8e03e40e8acf, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.582616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.583646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 999/1994] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:38 2022] DELETE /identity/v3/users/0f6923b6655e451fa06edeccd25979c6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:38.602758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 996/1995] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:38 2022] PUT /identity/v3/projects/8fa410029b42486eb0b9228afc066ea1/users/090ccf06098744928d4d8e03e40e8acf/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:38.613887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.614167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.614417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.652329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.655344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] MFA Rules not processed for user `090ccf06098744928d4d8e03e40e8acf`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:38.656741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.656894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.759988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1000/1996] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:38.761560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.765182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.776600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e2de-d4fb-4225-8cfd-c6988a381abc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.789547 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.790958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.791242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.791739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] PATH_INFO: `/v3/projects/a5244d8987ce42c990640b4bf903f0d8` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.792643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27003530-49b9-41ad-a2fe-6031948a7442 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.793421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5244d8987ce42c990640b4bf903f0d8)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.796854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.799436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:38.800295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.800511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.801029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.801693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:38.803708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:38.840081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1001/1997] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:38 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:38.843713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:38.854554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:38.878730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5244d8987ce42c990640b4bf903f0d8', 'request_id': 'req-2119e879-791a-42a6-8dc3-bbb4213b1337'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.886449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5244d8987ce42c990640b4bf903f0d8', 'request_id': 'req-2119e879-791a-42a6-8dc3-bbb4213b1337'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.898466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5244d8987ce42c990640b4bf903f0d8', 'request_id': 'req-2119e879-791a-42a6-8dc3-bbb4213b1337'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.919261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] The token cache is being invalidate because project a5244d8987ce42c990640b4bf903f0d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:38.919798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.920711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2119e879-791a-42a6-8dc3-bbb4213b1337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:38.922681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 997/1998] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:38 2022] DELETE /identity/v3/projects/a5244d8987ce42c990640b4bf903f0d8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:38.932799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:38.933803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:38.934103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:38.938709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a5c0-e28c-448b-8ba7-78f843d87a37 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:38.965990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:38.968232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:38.993845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.042728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 998/1999] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:39.044603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.046708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.068258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afc9bb-c836-4988-ba27-b37ca446df05 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.071064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.071930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.072375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.072714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.073423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.073834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.074794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.075137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.075343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24951a-bb88-476e-b283-eab85c551a9e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.075495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] PATH_INFO: `/v3/projects/ebcec72e0b7645acb6d86fbaf1d974ef` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.076819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcec72e0b7645acb6d86fbaf1d974ef)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.079357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.102533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1002/2000] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:38 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:39.110388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.110990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.112677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.145619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:39.147595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.151193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcec72e0b7645acb6d86fbaf1d974ef', 'request_id': 'req-0a151d9f-6652-4be7-99a7-87deee51b68e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.158136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcec72e0b7645acb6d86fbaf1d974ef', 'request_id': 'req-0a151d9f-6652-4be7-99a7-87deee51b68e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.165315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcec72e0b7645acb6d86fbaf1d974ef', 'request_id': 'req-0a151d9f-6652-4be7-99a7-87deee51b68e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.181019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] The token cache is being invalidate because project ebcec72e0b7645acb6d86fbaf1d974ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:39.181876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.182809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.185152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 999/2001] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:39 2022] DELETE /identity/v3/projects/ebcec72e0b7645acb6d86fbaf1d974ef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:39.186135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.187907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.230171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1003/2002] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:39.230171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.230654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.244575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66bd80-b8c0-4f37-bbef-7c9b23c9efdd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.250106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.250581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.251478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.251754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.252077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] PATH_INFO: `/v3/users/2d338ad4ef764f23818378a28cec7467` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.253277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d338ad4ef764f23818378a28cec7467)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.255419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.278486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a151d9f-6652-4be7-99a7-87deee51b68e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.300617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.301311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.301589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.301859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.305772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.308588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.314844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d338ad4ef764f23818378a28cec7467'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.330182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d338ad4ef764f23818378a28cec7467'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.331440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d338ad4ef764f23818378a28cec7467'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.334629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1000/2003] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:39.337043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.344492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d338ad4ef764f23818378a28cec7467'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.357696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d338ad4ef764f23818378a28cec7467'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.359418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576419b-2d04-492b-92e4-36b39763d271 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.366829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1004/2004] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:39 2022] DELETE /identity/v3/users/2d338ad4ef764f23818378a28cec7467 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:39.368443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.371116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.371381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.372190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.372471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.372744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.373477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:39.373833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.375629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.393231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1001/2005] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:39.394169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.398125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.420492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.421380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a606-9c83-457c-befe-b5cb27982474 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.461441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26d582-235c-417f-9311-1d162c1c7e56 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.468886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.471659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.471971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.472260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] PATH_INFO: `/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82/users/07bbab4d5bcc403cb03b38a086e4c8ce/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.499449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab7b30aeb6248859fc325f1f5b30b82, user_id=07bbab4d5bcc403cb03b38a086e4c8ce, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.508082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.541427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.542112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.542392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.542664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.543368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.544454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1005/2006] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:39 2022] PUT /identity/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82/users/07bbab4d5bcc403cb03b38a086e4c8ce/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:39.548230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.548431 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.594731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1002/2007] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:39 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:39.596416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.691663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff42df-cd4c-4293-8843-e6da296d6542 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.706732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.707866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.708024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.708327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.709070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.710723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.712282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424bc5-cbba-40a8-8960-665fbf1b2432 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.723120 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.723791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.724109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.724386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.725095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.727050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.736893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1006/2008] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:39.738067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.743439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.761595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178eec41-be12-4cd3-8a2e-49a94291c788 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.769014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.769971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.770287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.770552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] PATH_INFO: `/v3/projects/689823c80b954defa28f85c6ef1333a1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.772094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] RBAC: Authorizing `identity:delete_project(project_id=689823c80b954defa28f85c6ef1333a1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.774104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.783941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1003/2009] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:39 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:39.785589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.812318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eedb30-1bb7-4661-a928-8f2372c4be85 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.824355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.826927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.827917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.828617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] PATH_INFO: `/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82/users/07bbab4d5bcc403cb03b38a086e4c8ce/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.832956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab7b30aeb6248859fc325f1f5b30b82, user_id=07bbab4d5bcc403cb03b38a086e4c8ce, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.837660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.860803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1004/2010] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:39 2022] PUT /identity/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82/users/07bbab4d5bcc403cb03b38a086e4c8ce/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:39.860803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:39.868861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689823c80b954defa28f85c6ef1333a1', 'request_id': 'req-2da813ab-d005-4e16-821f-6ed4556d048f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.877210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689823c80b954defa28f85c6ef1333a1', 'request_id': 'req-2da813ab-d005-4e16-821f-6ed4556d048f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.886412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689823c80b954defa28f85c6ef1333a1', 'request_id': 'req-2da813ab-d005-4e16-821f-6ed4556d048f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.902531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.910586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] The token cache is being invalidate because project 689823c80b954defa28f85c6ef1333a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:39.911377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.912340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2da813ab-d005-4e16-821f-6ed4556d048f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:39.914252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1007/2011] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:39 2022] DELETE /identity/v3/projects/689823c80b954defa28f85c6ef1333a1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:39.915712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42972503-9197-4817-a74f-7a2df1f4932b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.916328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42972503-9197-4817-a74f-7a2df1f4932b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.916774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42972503-9197-4817-a74f-7a2df1f4932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.946652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-42972503-9197-4817-a74f-7a2df1f4932b None None] MFA Rules not processed for user `07bbab4d5bcc403cb03b38a086e4c8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:39.948069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-42972503-9197-4817-a74f-7a2df1f4932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:39.965518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce029bb2-8a28-4e8a-95dc-6f4b3dcf30a6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:39.970806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:39.971507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.971787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.972093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:39.972835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:39.974483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272e34f-c3b7-4c21-97e2-214f3592a184 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:39.975353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:39.989734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1005/2012] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:39.994099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:39.994871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:39.995473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.018971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.020383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.028652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1006/2013] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:40.029539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.033955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1008/2014] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:40.036593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.037040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.037222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.037515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.049941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec583a-04c4-41ae-b2f7-722b32dcf211 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.057139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.057820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.058099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.058370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.059112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.060769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.066773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.068189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.072596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1007/2015] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:40.073352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.077314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.080555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1009/2016] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:40.081149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.091281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407057f-fea8-4da5-91c8-0991ff947181 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.096550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.097533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.097879 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.098216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] PATH_INFO: `/v3/projects/3fd694376d3e43ddaa260d2ab48ee27c/users/9148132d72db4c65af017e61d896004c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.114870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd694376d3e43ddaa260d2ab48ee27c, user_id=9148132d72db4c65af017e61d896004c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.117846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.132615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1010/2017] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:40 2022] PUT /identity/v3/projects/3fd694376d3e43ddaa260d2ab48ee27c/users/9148132d72db4c65af017e61d896004c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:40.133635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.147399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d3ab8-13ce-4e08-95fb-ce1b8554ebcf admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.199769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.200459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.200774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.201054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.210972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4450-c4bd-4c6c-94bc-ff2bc8d6c90b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.215998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.216735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.217081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.217417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.218290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:40.218692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.220405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2603f8-9815-40c0-84cf-411a499c1fec admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.233834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1011/2018] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:40.235546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.235818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.236213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.263654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] MFA Rules not processed for user `9148132d72db4c65af017e61d896004c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.265074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.270685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.272280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.283897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1008/2019] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:40.284819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.300887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab840e2-8a56-4d66-a7a7-a294144a4291 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.310126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.310880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.311088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.311510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.312500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.314580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.319412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.351912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1009/2020] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:40.357150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.358421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1012/2021] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:40.362636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.374204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95bdbf-aca0-4d73-89a6-9866b80cd030 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.386797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.448322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d3f8-280f-4f35-8f79-a73fdf54a534 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.461376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.462088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.462375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.462659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.463375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.466144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.470763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.471456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.471739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.472115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.472818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.480491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.515797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1010/2022] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:40.518832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.529027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1013/2023] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:40.530150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.555012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d649-4b1f-4e4f-ac24-93a16823d1cb admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.631110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2341-2448-4063-b45a-e46c75b5bbdf admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.642150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.643848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.644196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.644483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.645191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.647669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.655349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.656730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.657106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.657504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.658354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.660099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.676923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1011/2024] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:40.680349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.680485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1014/2025] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:40.694297 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a2fec-b754-4123-8fc9-4ce21d86608d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.696508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.707627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.708359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.708638 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.708916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.709685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.711259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.717188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9584c3-d494-4e1a-9552-029af5199a83 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.724616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.725304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.726412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.726709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.727445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.727898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1012/2026] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:40.729284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.731752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.747685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1015/2027] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:40.753827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.755470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a31ff1-a324-476b-a443-ee5082a81c09 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.760761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.761671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.761965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.762178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] PATH_INFO: `/v3/projects/a64f28d4ec044d54b61672f9bb89efff/users/edec8c6a6ee84447825519b3000daac3/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.775966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb574-1d37-485c-9b1d-ca63c2df6125 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.784953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.785939 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.786170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.786544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] PATH_INFO: `/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd/users/ef1dde9220ac4a5cbdc15625bce69cc7/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.789967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64f28d4ec044d54b61672f9bb89efff, user_id=edec8c6a6ee84447825519b3000daac3, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.793798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a598a6-46ba-46d9-ac49-83eb97cd08b7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.807737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfd9d7d6465424eb7101e1d25ad27fd, user_id=ef1dde9220ac4a5cbdc15625bce69cc7, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.810736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.813779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1013/2028] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:40 2022] PUT /identity/v3/projects/a64f28d4ec044d54b61672f9bb89efff/users/edec8c6a6ee84447825519b3000daac3/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:40.826202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.826596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.827154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.843131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1016/2029] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:40 2022] PUT /identity/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd/users/ef1dde9220ac4a5cbdc15625bce69cc7/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:40.844375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.869500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] MFA Rules not processed for user `edec8c6a6ee84447825519b3000daac3`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:40.870820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.904726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.917646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:40.939566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc91fe4-5e3a-43dd-8ca7-df9f0bf25210 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.952061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.952883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.953163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.953436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.955458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.956016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.969684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1014/2030] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:40.969684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.969940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1017/2031] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:40.972513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:40.972685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:40.984690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67690e20-e89f-4d80-b120-bff8a3fc928d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.990292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:40.990728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:40.991014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:40.991263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb82fc-20c9-4bf1-a987-d86608f0fad4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:40.991432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:40.992252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:40.994460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:40.998995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.000188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.000708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.001157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] PATH_INFO: `/v3/users/090ccf06098744928d4d8e03e40e8acf` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.003570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090ccf06098744928d4d8e03e40e8acf)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.006871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.027500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1018/2032] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:40 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:41.028663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.073740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090ccf06098744928d4d8e03e40e8acf'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:41.092105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090ccf06098744928d4d8e03e40e8acf'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:41.094225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090ccf06098744928d4d8e03e40e8acf'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:41.104489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090ccf06098744928d4d8e03e40e8acf'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:41.108519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ee9a6-9a57-4846-9ccf-c18ec2716be8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.115703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.116351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090ccf06098744928d4d8e03e40e8acf'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:41.116806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.117086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.117506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] PATH_INFO: `/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd/users/ef1dde9220ac4a5cbdc15625bce69cc7/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.121500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfd9d7d6465424eb7101e1d25ad27fd, user_id=ef1dde9220ac4a5cbdc15625bce69cc7, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.123849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ce28d-32fc-4536-a641-3f7382a1538d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.124183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1015/2033] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:40 2022] DELETE /identity/v3/users/090ccf06098744928d4d8e03e40e8acf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:41.127918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.128666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.160181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1019/2034] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:41 2022] PUT /identity/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd/users/ef1dde9220ac4a5cbdc15625bce69cc7/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:41.160181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a003f-6761-463d-8e88-193e17dd8d26 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.160181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a003f-6761-463d-8e88-193e17dd8d26 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.160181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a003f-6761-463d-8e88-193e17dd8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.183406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-de6a003f-6761-463d-8e88-193e17dd8d26 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:41.184726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-de6a003f-6761-463d-8e88-193e17dd8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.234276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1eb9-d2c9-4f40-aad3-7ce71a307ca3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.259788 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1020/2035] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:41.261291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.261676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.262059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.289127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] MFA Rules not processed for user `ef1dde9220ac4a5cbdc15625bce69cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:41.289560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.290460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.304757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.305515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.305861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.306300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.307003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.309014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.330025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1016/2036] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:41 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:41.331016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.332740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.344975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.368783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1021/2037] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:41.369621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.381435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ac064-cdfe-48da-a249-54bf44fc3f58 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.385979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.386640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.386916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.387185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.388120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.389881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.396200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c625745-1f40-4286-927a-36f3a86eb947 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.400230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.400733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1022/2038] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:41.400851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.401128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.401369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.401634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.403432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.411360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.412818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.422552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1017/2039] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:41.423401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.433934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca5f58-38a7-4431-9332-c2959c306a72 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.440817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.441719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.441993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.442888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] PATH_INFO: `/v3/projects/09e220f7e6e6415e92478874ed8e7421/users/94cebc244c0441f08c277002916b55a3/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.459090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e220f7e6e6415e92478874ed8e7421, user_id=94cebc244c0441f08c277002916b55a3, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.461766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.470281 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fe277-81fc-432e-ae46-5f43123fa00a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.476486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1018/2040] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:41 2022] PUT /identity/v3/projects/09e220f7e6e6415e92478874ed8e7421/users/94cebc244c0441f08c277002916b55a3/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:41.480691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.569715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.570714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.571209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.571678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.583699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400abff8-fb3b-4310-b722-a7aa52187aa6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.590481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.591400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.591793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.592189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.593548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.596119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.608566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1019/2041] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:41.612895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.626575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a110afd-646f-46cc-bf29-d65664910162 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.634509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.635831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.636241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.637135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] PATH_INFO: `/v3/projects/09e220f7e6e6415e92478874ed8e7421/users/94cebc244c0441f08c277002916b55a3/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.639691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e220f7e6e6415e92478874ed8e7421, user_id=94cebc244c0441f08c277002916b55a3, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.643289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce7a4b-2f57-4159-a09d-b2b1ab04e25d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.657867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1020/2042] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:41 2022] PUT /identity/v3/projects/09e220f7e6e6415e92478874ed8e7421/users/94cebc244c0441f08c277002916b55a3/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:41.672989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.673275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.673542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.705674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.707253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.719368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] MFA Rules not processed for user `94cebc244c0441f08c277002916b55a3`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:41.720294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1023/2043] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:41.722784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.778228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.795381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.797899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:41.822045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688f381-429f-4b77-a8a1-11fc7cdb3425 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.885311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1021/2044] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:41.893549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:41.901531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:41.929297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.930400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.930845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.931260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] PATH_INFO: `/v3/projects/8fa410029b42486eb0b9228afc066ea1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.933137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa410029b42486eb0b9228afc066ea1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:41.936230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:41.974363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bec4e-2b2d-42c1-92cf-ba6cbf31bf81 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:41.978692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:41.979528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:41.979995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:41.980436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:41.982557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.003497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa410029b42486eb0b9228afc066ea1', 'request_id': 'req-5409df3d-6b05-4843-8c7e-2a307f59c7cf'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.012123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa410029b42486eb0b9228afc066ea1', 'request_id': 'req-5409df3d-6b05-4843-8c7e-2a307f59c7cf'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.020382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa410029b42486eb0b9228afc066ea1', 'request_id': 'req-5409df3d-6b05-4843-8c7e-2a307f59c7cf'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.036490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] The token cache is being invalidate because project 8fa410029b42486eb0b9228afc066ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:42.037027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.038635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.040878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1024/2045] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:41 2022] DELETE /identity/v3/projects/8fa410029b42486eb0b9228afc066ea1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:42.045209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.047087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.066001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.067449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.078377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1022/2046] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:42.131981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409df3d-6b05-4843-8c7e-2a307f59c7cf admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.137403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.139322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.139322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.139322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] PATH_INFO: `/v3/users/07bbab4d5bcc403cb03b38a086e4c8ce` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.141503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bbab4d5bcc403cb03b38a086e4c8ce)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.148558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.202117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bbab4d5bcc403cb03b38a086e4c8ce'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.215233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bbab4d5bcc403cb03b38a086e4c8ce'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.216144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bbab4d5bcc403cb03b38a086e4c8ce'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.223615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.227778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bbab4d5bcc403cb03b38a086e4c8ce'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.237743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bbab4d5bcc403cb03b38a086e4c8ce'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.247566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1025/2047] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:42 2022] DELETE /identity/v3/users/07bbab4d5bcc403cb03b38a086e4c8ce => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:42.621770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.643116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac8bc9-19be-4f05-9c3b-52be7d92f8b5 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.731824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.732940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.733699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.734820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] PATH_INFO: `/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.736228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab7b30aeb6248859fc325f1f5b30b82)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.738576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.766416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:42.796594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:42.798276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.804715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab7b30aeb6248859fc325f1f5b30b82', 'request_id': 'req-519228c7-f8ae-4cfc-8649-8b223181b52b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.810252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab7b30aeb6248859fc325f1f5b30b82', 'request_id': 'req-519228c7-f8ae-4cfc-8649-8b223181b52b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.813745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33a44-35d9-4a2a-bcef-6046964dd131 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:42.816695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab7b30aeb6248859fc325f1f5b30b82', 'request_id': 'req-519228c7-f8ae-4cfc-8649-8b223181b52b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.832044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] The token cache is being invalidate because project 8ab7b30aeb6248859fc325f1f5b30b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:42.832355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.832982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-519228c7-f8ae-4cfc-8649-8b223181b52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.834882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1023/2048] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:42 2022] DELETE /identity/v3/projects/8ab7b30aeb6248859fc325f1f5b30b82 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:42.839475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eaf51-ae0e-4971-9603-d335ab4273f6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.839755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eaf51-ae0e-4971-9603-d335ab4273f6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.840063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eaf51-ae0e-4971-9603-d335ab4273f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.868767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d99eaf51-ae0e-4971-9603-d335ab4273f6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:42.870648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d99eaf51-ae0e-4971-9603-d335ab4273f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:42.883930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:42.884981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.885265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.885917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] PATH_INFO: `/v3/users/9148132d72db4c65af017e61d896004c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.887329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9148132d72db4c65af017e61d896004c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:42.890844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:42.960988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9148132d72db4c65af017e61d896004c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.973301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1024/2049] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:42.975782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:42.976273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:42.976553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:42.981567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9148132d72db4c65af017e61d896004c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.982323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9148132d72db4c65af017e61d896004c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.990748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9148132d72db4c65af017e61d896004c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:42.998917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:43.000197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.005020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-86fe7e01-e77e-4930-b8f3-b63c1b3f8d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9148132d72db4c65af017e61d896004c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.008600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1025/2050] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:43.012850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.016148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1026/2051] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:42 2022] DELETE /identity/v3/users/9148132d72db4c65af017e61d896004c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:43.021275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.022592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.022998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.025470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e96876-ce21-4b4d-a2cc-82b6a62007a6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.044280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:43.045618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.094142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.095417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.095721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.096034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.105545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:43.105545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.105545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.112536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1026/2052] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:43.117140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.118920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.128234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1027/2053] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:43.133165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3b37b-337d-4450-9ca0-3e39350b478a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.138021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.138343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.138811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.139156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.139540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.140113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.140455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.142432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.152160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.173078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1027/2054] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:43.177381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.196273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.208428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ffe4-2cb7-4d7a-997c-2ec95f101586 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.245876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6638-9f7a-45ed-9d08-f5775779d9c0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.251789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.252501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.252783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.253057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.253755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.255683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.265100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.265757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.266103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.266380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.275911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.276035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1028/2055] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:43.277354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.280146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.287032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1028/2056] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:43.289820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee32751-0f55-46e8-bc74-01a1637cff80 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.294502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.295037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.295539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.295998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.296752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.298379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.309359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1029/2057] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:43.313516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.323827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61284745-a31d-4ae6-be15-788c600938ba service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.329427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.330196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.330584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.330932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.331736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.333511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658c89d-3155-4900-9093-b8a90282c8ba admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.345764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1029/2058] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:43.348664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.359238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4a2bb-9131-412a-9ece-51f409035e20 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.364501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.365560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.365900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.366248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] PATH_INFO: `/v3/projects/5428226a08ef4ae98c074cf1c54c7b2b/users/e1dbc9149b9846f39fff34c7bdcaf4f3/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.381910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5428226a08ef4ae98c074cf1c54c7b2b, user_id=e1dbc9149b9846f39fff34c7bdcaf4f3, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.384622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.398765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1030/2059] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:43 2022] PUT /identity/v3/projects/5428226a08ef4ae98c074cf1c54c7b2b/users/e1dbc9149b9846f39fff34c7bdcaf4f3/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:43.409544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.409884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.410107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.442731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] MFA Rules not processed for user `e1dbc9149b9846f39fff34c7bdcaf4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:43.444017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.508700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1030/2060] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:43.513283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.566016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.567455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.601542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495a9-9496-4ca7-9742-be5e834e74a5 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.607391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.608532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.608820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.609165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.610030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.612983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.628896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.647543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1031/2061] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:43.653140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.658992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93603426-5576-4693-8dcf-72f2f5ac7007 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.732227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:43.747554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.749092 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.749377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.749653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] PATH_INFO: `/v3/projects/3fd694376d3e43ddaa260d2ab48ee27c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.750472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72162b-9da3-480a-aedb-3f3345d9f235 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.750972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd694376d3e43ddaa260d2ab48ee27c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.752946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.755169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.755839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.756153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.756437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.757140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.759200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.784684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1032/2062] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:43 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:43.789252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.801379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebd353-ce59-4291-b05d-beb05dff29c7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.817977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd694376d3e43ddaa260d2ab48ee27c', 'request_id': 'req-c90f0243-1d01-463a-9a79-add48dd0ead3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.824500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd694376d3e43ddaa260d2ab48ee27c', 'request_id': 'req-c90f0243-1d01-463a-9a79-add48dd0ead3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.831807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd694376d3e43ddaa260d2ab48ee27c', 'request_id': 'req-c90f0243-1d01-463a-9a79-add48dd0ead3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.847671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] The token cache is being invalidate because project 3fd694376d3e43ddaa260d2ab48ee27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:43.848067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.848959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:43.850752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1031/2063] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:43 2022] DELETE /identity/v3/projects/3fd694376d3e43ddaa260d2ab48ee27c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:43.854754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.856466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.867709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.868536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.868918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.869211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.870096 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.871728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.882860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1033/2064] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:43.887012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.888792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:43.902360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7440f-02c4-4ee5-a91b-48dd5b717956 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.909136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.910171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.910458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.911215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] PATH_INFO: `/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d/users/58580bf0e646460d956ece63f82be617/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.933728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0243-1d01-463a-9a79-add48dd0ead3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:43.940618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fd96be302b4ae6bafc1b5a4d9fe39d, user_id=58580bf0e646460d956ece63f82be617, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.941738 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:43.942724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:43.942902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:43.943167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] PATH_INFO: `/v3/users/ef1dde9220ac4a5cbdc15625bce69cc7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:43.943528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.944390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1dde9220ac4a5cbdc15625bce69cc7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:43.946408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:43.968795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1034/2065] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:43 2022] PUT /identity/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d/users/58580bf0e646460d956ece63f82be617/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:43.973966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:43.986566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.016848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1dde9220ac4a5cbdc15625bce69cc7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.028131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1dde9220ac4a5cbdc15625bce69cc7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.028977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1dde9220ac4a5cbdc15625bce69cc7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.037126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1dde9220ac4a5cbdc15625bce69cc7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.046583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1dde9220ac4a5cbdc15625bce69cc7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.053530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1032/2066] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:43 2022] DELETE /identity/v3/users/ef1dde9220ac4a5cbdc15625bce69cc7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.061637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da39fb6-270f-4fa8-9ded-c9129ba52b9e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.124045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.124751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.124988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.125261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.126112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.128508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ed2c2-3fbe-46eb-a840-ea6de1346163 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.131290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.147274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1035/2067] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:44.147881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.149677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.162937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dcabe-d08c-444c-bc04-645eaf6c71a1 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.167717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.168724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.168984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.169261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] PATH_INFO: `/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d/users/58580bf0e646460d956ece63f82be617/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.171396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fd96be302b4ae6bafc1b5a4d9fe39d, user_id=58580bf0e646460d956ece63f82be617, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.174370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.186165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1033/2068] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:44 2022] PUT /identity/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d/users/58580bf0e646460d956ece63f82be617/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.196970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.197264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.197548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.227803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] MFA Rules not processed for user `58580bf0e646460d956ece63f82be617`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:44.229066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.317981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1036/2069] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:44.325980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.328243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.403052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587d6d-370b-434e-a8e2-f708cfbbe349 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.409050 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.409744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.410504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.410853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.411230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.411634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.432800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.434858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f13c-1876-49ef-a90f-9826f63a6551 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.444703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1034/2070] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:44.498200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e0ca-d95d-4150-b4da-e5b4712d7381 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.503294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.504366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.504655 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.504958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] PATH_INFO: `/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.506159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfd9d7d6465424eb7101e1d25ad27fd)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.508118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.555252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfd9d7d6465424eb7101e1d25ad27fd', 'request_id': 'req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.562521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfd9d7d6465424eb7101e1d25ad27fd', 'request_id': 'req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.573663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfd9d7d6465424eb7101e1d25ad27fd', 'request_id': 'req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.589853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] The token cache is being invalidate because project bdfd9d7d6465424eb7101e1d25ad27fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:44.590188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.590850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e07f7aac-13ca-45bc-b326-87eaa75ff1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:44.592764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1037/2071] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:44 2022] DELETE /identity/v3/projects/bdfd9d7d6465424eb7101e1d25ad27fd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:44.599630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.599965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.600247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.618933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:44.620420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.681051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1035/2072] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:44.689843 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.690349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.690349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.711788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:44.713023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.721644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1038/2073] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:44.726117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.736925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6836ea-9c41-4997-8756-a10a2227ea3a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.742708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.743403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.743687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.743995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.744739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:44.745045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.746659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.759661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1036/2074] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:44.765077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.774980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.776216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0aab5d-8475-46e5-a416-5ea237c3eb88 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.776866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:44.781141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.781804 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.782083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.782355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.783047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.785008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.789555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.806933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:44.815435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1039/2075] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:44.820800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.847785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef872c-f48a-4212-9599-d53fc5926b7c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.882173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf542eb-d465-49ce-9af1-d3f100e2dcda admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.886768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.887438 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.887722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.888055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.888755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.890791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.904854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58580bf0e646460d956ece63f82be617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.905633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.905900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.906170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.906906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:44.907215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.908861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.918617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1040/2076] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:44.918617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.927844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1037/2077] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:44.928924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.934736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aa972-95c1-4c3e-bab1-05f9da1d8de2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.939562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9821f-3af9-4f5b-81b9-ce582327fdac tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.941058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.942128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.943102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.943102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.944442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58580bf0e646460d956ece63f82be617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:44.944783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.945108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:44.945399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:44.945684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:44.946377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:44.947019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.948314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:44.960656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1041/2078] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:44.965440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:44.976134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1038/2079] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:44 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:44.977683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f25d1-7175-49d6-bb13-5651dd34b048 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:44.980083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.044156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50a42-72e2-4ce4-aefe-5b3a473a7008 tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.046419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.047180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.047595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.047976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.048786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.049380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58580bf0e646460d956ece63f82be617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.050225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.050580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.050768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.050985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.051780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:45.052185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.053871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.061980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1042/2080] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:45.066551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1039/2081] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:45.070799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.081323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a38b-f4f4-4ed5-ab55-21f8816d194e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.086023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.086912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.087194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.087470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] PATH_INFO: `/v3/projects/47872575b3ea451ca932797b667453cc/users/cf0c8fbf973444edbc97edb089f7e2b9/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.102886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.103166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47872575b3ea451ca932797b667453cc, user_id=cf0c8fbf973444edbc97edb089f7e2b9, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.105879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0ed9e-c098-468c-b179-eeea3897a43f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.113007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7683a-24ec-4536-a30a-0fd600c9ac0a tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.119279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1043/2082] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:45 2022] PUT /identity/v3/projects/47872575b3ea451ca932797b667453cc/users/cf0c8fbf973444edbc97edb089f7e2b9/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:45.130168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.130534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.130881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.144684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.157997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] MFA Rules not processed for user `cf0c8fbf973444edbc97edb089f7e2b9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:45.159205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.171742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58580bf0e646460d956ece63f82be617', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.172775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.173136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.173487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] PATH_INFO: `/v3/projects/f9053beb2bd04bd3a0950ecfe2682fbe` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.174781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f9053beb2bd04bd3a0950ecfe2682fbe)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.176881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.182670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.225202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1044/2083] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:45.227067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9053beb2bd04bd3a0950ecfe2682fbe', 'request_id': 'req-0b6a8233-a100-420e-895b-07a3e200fd8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.233811 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9053beb2bd04bd3a0950ecfe2682fbe', 'request_id': 'req-0b6a8233-a100-420e-895b-07a3e200fd8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.236931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.238388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.240268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9053beb2bd04bd3a0950ecfe2682fbe', 'request_id': 'req-0b6a8233-a100-420e-895b-07a3e200fd8d'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.253793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] The token cache is being invalidate because project f9053beb2bd04bd3a0950ecfe2682fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:45.254082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.254731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.256324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1040/2084] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:45 2022] DELETE /identity/v3/projects/f9053beb2bd04bd3a0950ecfe2682fbe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:45.299665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842bd9f-5198-4c31-92e0-0db04e51b58e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.353219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.354163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.354612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.356314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.358435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.426977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.428305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc0c69-f2fe-4d8f-880b-394f460aa8e7 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.438772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1045/2085] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:45.450070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.451918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.540054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a8233-a100-420e-895b-07a3e200fd8d tempest-VolumeQuotaClassesTest-1737726817 tempest-VolumeQuotaClassesTest-1737726817-project-admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.544908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.545920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.546323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.546626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] PATH_INFO: `/v3/users/e1dbc9149b9846f39fff34c7bdcaf4f3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.548318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dbc9149b9846f39fff34c7bdcaf4f3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.550528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.620941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dbc9149b9846f39fff34c7bdcaf4f3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.638020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dbc9149b9846f39fff34c7bdcaf4f3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.638869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dbc9149b9846f39fff34c7bdcaf4f3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.640698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.652711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dbc9149b9846f39fff34c7bdcaf4f3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.667106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dbc9149b9846f39fff34c7bdcaf4f3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:45.673626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1041/2086] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:45 2022] DELETE /identity/v3/users/e1dbc9149b9846f39fff34c7bdcaf4f3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:45.678069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.678364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.678700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.695766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:45.696998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.729421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:45.766854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1046/2087] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:45.777101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:45.778632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:45.851991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf5e-a773-4a36-8cf2-bb1cd1d0e8b0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:45.861188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:45.862196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:45.862934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:45.863214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:45.880008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:45.881827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:45.894150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1042/2088] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:46.195984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.197645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:46.211197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2d456-d58a-417e-bfae-a95cd0505c5f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.216987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.218209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.220180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.220180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] PATH_INFO: `/v3/projects/5428226a08ef4ae98c074cf1c54c7b2b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.220180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5428226a08ef4ae98c074cf1c54c7b2b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.222033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.270808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5428226a08ef4ae98c074cf1c54c7b2b', 'request_id': 'req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.276295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5428226a08ef4ae98c074cf1c54c7b2b', 'request_id': 'req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.282579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5428226a08ef4ae98c074cf1c54c7b2b', 'request_id': 'req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.295405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] The token cache is being invalidate because project 5428226a08ef4ae98c074cf1c54c7b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:46.295632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.296303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.297883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1047/2089] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:46 2022] DELETE /identity/v3/projects/5428226a08ef4ae98c074cf1c54c7b2b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:46.302121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.312088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ceb6-2095-4729-bd4f-0813063b35ac service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:46.371787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:46.372706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:46.372984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:46.373239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] PATH_INFO: `/v3/users/58580bf0e646460d956ece63f82be617` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:46.374565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58580bf0e646460d956ece63f82be617)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:46.376582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:46.427814 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58580bf0e646460d956ece63f82be617'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.440650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58580bf0e646460d956ece63f82be617'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.441323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58580bf0e646460d956ece63f82be617'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.448347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58580bf0e646460d956ece63f82be617'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.457023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-06cecc04-081d-485e-92c9-cf141b153e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58580bf0e646460d956ece63f82be617'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:46.463364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1043/2090] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:46 2022] DELETE /identity/v3/users/58580bf0e646460d956ece63f82be617 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:46.911705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:46.913493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:46.929166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e98cf3-ee61-45d7-bb51-ce8f5d25f7a8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.044394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.045332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.045611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.045922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] PATH_INFO: `/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.047966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fd96be302b4ae6bafc1b5a4d9fe39d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.050014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.107553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'request_id': 'req-38aa0770-c597-42b4-bc00-28722d1ee596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:47.114966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'request_id': 'req-38aa0770-c597-42b4-bc00-28722d1ee596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:47.117137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:47.126262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fd96be302b4ae6bafc1b5a4d9fe39d', 'request_id': 'req-38aa0770-c597-42b4-bc00-28722d1ee596'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:47.150506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] The token cache is being invalidate because project f8fd96be302b4ae6bafc1b5a4d9fe39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:47.150622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:47.151336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-38aa0770-c597-42b4-bc00-28722d1ee596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:47.153078 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1048/2091] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:46 2022] DELETE /identity/v3/projects/f8fd96be302b4ae6bafc1b5a4d9fe39d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:47.160051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.160257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.160996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.170925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:47.213696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:47.214930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:47.300710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1044/2092] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:47.308649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269092-54ca-471b-87dd-0189e010805f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.309230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269092-54ca-471b-87dd-0189e010805f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.309517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269092-54ca-471b-87dd-0189e010805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.328501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-64269092-54ca-471b-87dd-0189e010805f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:47.329718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-64269092-54ca-471b-87dd-0189e010805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:47.339284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1049/2093] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:47.344503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.354333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9b3ac-9db4-4994-bfd6-6d78f877ab3f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.359731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.360711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.360827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.361084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.361821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:47.362133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.363752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.376834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1045/2094] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:47.381136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269092-54ca-471b-87dd-0189e010805f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.391349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269092-54ca-471b-87dd-0189e010805f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.396751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.397477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.397826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.399436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.399436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.400942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.433195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1050/2095] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:47.438453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.450961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563b0f-35bd-4c83-82e0-2bec12a34532 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.571287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.572283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.572630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.572976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.573772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.576219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.603615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1046/2096] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:47.607850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.621069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c65f1a-6333-4649-a26b-4b7a6094b2db admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.627343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.628702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.629171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.629522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.630331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.632126 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.646061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1051/2097] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:47.649920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.661413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf95d0-ce85-442b-85af-d3d37c7ee393 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.666524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.667303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.667647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.668018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.668855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.670819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41d776-0b7e-4749-bf60-446f2a43b1a0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.685430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1047/2098] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:47.686705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.697676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c026f0-d6d3-47f5-ada9-66b18e16172c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.702562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.703651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.703965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.704248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] PATH_INFO: `/v3/projects/00769abffe934ab39edb442f4050c5f2/users/cd7938bebed14826950475c0cd6c021b/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.722150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=00769abffe934ab39edb442f4050c5f2, user_id=cd7938bebed14826950475c0cd6c021b, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.725183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.742374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1052/2099] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:47 2022] PUT /identity/v3/projects/00769abffe934ab39edb442f4050c5f2/users/cd7938bebed14826950475c0cd6c021b/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:47.753004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.753309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.753618 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.782397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] MFA Rules not processed for user `cd7938bebed14826950475c0cd6c021b`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:47.783604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:47.845167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:47.849336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1048/2100] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:47.853883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.924552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96e6c6-a489-44ac-a0ce-0fcf1707a99c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:47.929355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:47.930048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:47.930330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:47.930607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:47.931319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:47.933250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:47.971036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1053/2101] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:47.975089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:47.987617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d382900-f477-4b6f-b7d6-1d6b2c6b60d0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.066922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.067688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.068070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.068413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.069187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.071288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.094627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1049/2102] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:47 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:48.098723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.111478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702b7d-4af4-4cc8-95ed-d195e4447bd8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.116428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.117100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.117382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.117658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.118519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.120237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.131781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1054/2103] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:48.136298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.146845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641514b-d0e0-4851-9bd6-e1510d81715a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.151778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.152836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.153150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.153427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] PATH_INFO: `/v3/projects/859c655e89b445dabcbe747d52fbb87f/users/b3b0e77671034224b1ad89cbb0d48db2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.169933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=859c655e89b445dabcbe747d52fbb87f, user_id=b3b0e77671034224b1ad89cbb0d48db2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.172956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.192068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1050/2104] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:48 2022] PUT /identity/v3/projects/859c655e89b445dabcbe747d52fbb87f/users/b3b0e77671034224b1ad89cbb0d48db2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:48.192166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.201360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2284ac1-489e-4ff3-b06a-80343f6382d3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.272597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.273278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.273563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.273838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.274573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.276208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e78cb-55f9-41ff-87c9-4f8be817df37 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.277480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.289345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1055/2105] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:48.293287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.306009 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e2dd4-cd7e-4e14-a11d-1debb2cf6bfb admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.310576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.311486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.311762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.312067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] PATH_INFO: `/v3/projects/859c655e89b445dabcbe747d52fbb87f/users/b3b0e77671034224b1ad89cbb0d48db2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.314010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859c655e89b445dabcbe747d52fbb87f, user_id=b3b0e77671034224b1ad89cbb0d48db2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.316683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.329140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1051/2106] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:48 2022] PUT /identity/v3/projects/859c655e89b445dabcbe747d52fbb87f/users/b3b0e77671034224b1ad89cbb0d48db2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:48.341281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.341723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.342008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.368514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] MFA Rules not processed for user `b3b0e77671034224b1ad89cbb0d48db2`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:48.369753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:48.446377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1056/2107] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:48.457460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.459158 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:48.523255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200a692-5d2a-4509-99ce-38b5f8829f85 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.526984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.527672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.528041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.528376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.538952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.541452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083130c-86d5-418c-be5f-752a3f1b1d6a service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.561459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1052/2108] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:48.682919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.750404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991e055-8dff-4b4d-b596-d90c89ffc158 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:48.755460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:48.756368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.756639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.756918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] PATH_INFO: `/v3/users/cd7938bebed14826950475c0cd6c021b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.758349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7938bebed14826950475c0cd6c021b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:48.760318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:48.828015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7938bebed14826950475c0cd6c021b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:48.840537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7938bebed14826950475c0cd6c021b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:48.841264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7938bebed14826950475c0cd6c021b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:48.848249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7938bebed14826950475c0cd6c021b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:48.858856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7938bebed14826950475c0cd6c021b'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:48.861699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.867053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1057/2109] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:48 2022] DELETE /identity/v3/users/cd7938bebed14826950475c0cd6c021b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:48.872243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:48.872469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:48.872748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:48.893699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:48.894904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:48.945668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:48.964005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1053/2110] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:48.974645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:48.976589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:49.042270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb60d4b-3300-46d4-8293-b5a5fe8e5fdc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.046138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.046749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.047002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.047309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.056609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.058046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.068161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1058/2111] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:49.406676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.408192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:49.420809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60657a-4fa2-4a26-bdbb-0ee3c0962313 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.426989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.427895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.428192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.428464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] PATH_INFO: `/v3/projects/00769abffe934ab39edb442f4050c5f2` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.429772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00769abffe934ab39edb442f4050c5f2)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.431786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.482959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00769abffe934ab39edb442f4050c5f2', 'request_id': 'req-164f4123-c9b9-407e-a910-0d0ced768b02'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.490186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00769abffe934ab39edb442f4050c5f2', 'request_id': 'req-164f4123-c9b9-407e-a910-0d0ced768b02'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.497836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00769abffe934ab39edb442f4050c5f2', 'request_id': 'req-164f4123-c9b9-407e-a910-0d0ced768b02'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.516484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] The token cache is being invalidate because project 00769abffe934ab39edb442f4050c5f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:49.516770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.517459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.519341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1054/2112] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:49 2022] DELETE /identity/v3/projects/00769abffe934ab39edb442f4050c5f2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:49.523501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.533815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4ee-3952-4b04-b416-86fc9a57b9be service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:49.593089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:49.593845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:49.594154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:49.594436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] PATH_INFO: `/v3/users/b3b0e77671034224b1ad89cbb0d48db2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:49.595686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b0e77671034224b1ad89cbb0d48db2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:49.597699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:49.646635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b0e77671034224b1ad89cbb0d48db2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.659946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b0e77671034224b1ad89cbb0d48db2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.660651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b0e77671034224b1ad89cbb0d48db2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.667508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b0e77671034224b1ad89cbb0d48db2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.677199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-cd0a9caa-b4f1-4b85-99e7-d523c6bf77c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b0e77671034224b1ad89cbb0d48db2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:49.684038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1059/2113] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:49 2022] DELETE /identity/v3/users/b3b0e77671034224b1ad89cbb0d48db2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:49.970710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:49.972227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:49.986597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f4123-c9b9-407e-a910-0d0ced768b02 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.052219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.053129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.053406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.053677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] PATH_INFO: `/v3/projects/859c655e89b445dabcbe747d52fbb87f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.054942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=859c655e89b445dabcbe747d52fbb87f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.056905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.103393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859c655e89b445dabcbe747d52fbb87f', 'request_id': 'req-256a6749-3764-4f92-8676-c1213ad60418'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:50.109856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859c655e89b445dabcbe747d52fbb87f', 'request_id': 'req-256a6749-3764-4f92-8676-c1213ad60418'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:50.116016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859c655e89b445dabcbe747d52fbb87f', 'request_id': 'req-256a6749-3764-4f92-8676-c1213ad60418'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:50.131263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] The token cache is being invalidate because project 859c655e89b445dabcbe747d52fbb87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:50.131671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:50.132319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-256a6749-3764-4f92-8676-c1213ad60418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:50.133878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1055/2114] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:49 2022] DELETE /identity/v3/projects/859c655e89b445dabcbe747d52fbb87f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:50.140524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.141008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.141230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.160529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:50.161814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.179477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.239803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1060/2115] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:50.247628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.248612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.248892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.266893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:50.268445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.276173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1056/2116] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:50.280394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.290608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc9c2-4dff-4309-960a-54a60b0cd801 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.295272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.296006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.296290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.296569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.297298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:50.297605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.299330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.312248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1061/2117] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:50.316973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.321431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.327171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4659ee4-d5b2-402a-a367-36d2b139a782 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.333743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.334407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.334969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.335310 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.336044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.338206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.366097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1057/2118] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:50.370896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.381611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3c9f-9f9b-4106-be6b-8cc93cf414aa admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.446069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.446876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.447054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.447993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.448684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.450846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.476900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1062/2119] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:50.478769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.491535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fe049-e1eb-41c7-85d6-a6817fe8a1ca admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.496880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.497563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.497853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.498130 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.498868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.501199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.516128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1058/2120] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:50.523631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.534288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d1468-e1df-4e1a-a2b2-fc10c6e7b923 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.540296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.541075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.541353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.541625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.542361 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.544011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302f328-8bb2-4ac0-97ea-3bcf6233c68e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.555105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1063/2121] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:50.559560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.570396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9f777-7171-4618-a18e-237d02ffb6a3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.575658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.576591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.576902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.577230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] PATH_INFO: `/v3/projects/d6934da15a2046908e0d8c5907d8a76b/users/b75dc6fd804448bdbc52a7b703614049/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.593150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6934da15a2046908e0d8c5907d8a76b, user_id=b75dc6fd804448bdbc52a7b703614049, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.595897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.610178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1059/2122] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:50 2022] PUT /identity/v3/projects/d6934da15a2046908e0d8c5907d8a76b/users/b75dc6fd804448bdbc52a7b703614049/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:50.620880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.621296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.621652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.661988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] MFA Rules not processed for user `b75dc6fd804448bdbc52a7b703614049`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:50.663378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.767631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1064/2123] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:50.812584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.814576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.849286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:50.853266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.878545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:50.883463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333629-d5ce-4e3e-a7f8-e75454cc6c85 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.887544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.888243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.888530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.888808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.898601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:50.900200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:50.909381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1060/2124] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:50.924427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f742a7c-9360-4a30-ab2a-46f2e9f7c43d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:50.928298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:50.928922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:50.929209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:50.929486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:50.931493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:50.999388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:51.000929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e468e7-6490-40ee-8a59-dd601f944777 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:51.011537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1065/2125] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:54.102067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.104698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.173718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a483d7-f1c9-4e4e-8cea-02dc150ff2f9 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.178238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.179113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.179412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.179652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] PATH_INFO: `/v3/users/cf0c8fbf973444edbc97edb089f7e2b9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.187516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0c8fbf973444edbc97edb089f7e2b9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.189702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.243761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0c8fbf973444edbc97edb089f7e2b9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.246134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.261368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0c8fbf973444edbc97edb089f7e2b9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.262256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0c8fbf973444edbc97edb089f7e2b9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.281847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0c8fbf973444edbc97edb089f7e2b9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.290112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0c8fbf973444edbc97edb089f7e2b9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.299891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1061/2126] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:54 2022] DELETE /identity/v3/users/cf0c8fbf973444edbc97edb089f7e2b9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:54.304651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.305170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.305261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.322538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:54.323912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.394281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1066/2127] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:54.404720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.406552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.468754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.469339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae44ec2-4e7a-4bbe-8b98-39087d774f7e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.470521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.475150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.475799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.476116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.476394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.484401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05ac7f-c125-49e3-b29b-5ae5fa6234fb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.486159 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.487591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e00ef-10b7-46bc-8584-21f71f508d2e service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.490359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.491246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.491513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.491795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] PATH_INFO: `/v3/users/b75dc6fd804448bdbc52a7b703614049` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.494295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75dc6fd804448bdbc52a7b703614049)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.496672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.498757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1062/2128] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:54.575714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75dc6fd804448bdbc52a7b703614049'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.591177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75dc6fd804448bdbc52a7b703614049'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.592353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.593373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75dc6fd804448bdbc52a7b703614049'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.603705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75dc6fd804448bdbc52a7b703614049'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.613439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75dc6fd804448bdbc52a7b703614049'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.622307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1067/2129] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:54 2022] DELETE /identity/v3/users/b75dc6fd804448bdbc52a7b703614049 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:54.627494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.628088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.628784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.649872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:54.650647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.761142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1063/2130] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:54.777221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.779437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.826668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:54.828966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:54.854502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52a9c3-6b45-4f24-84aa-2b652ac01e0f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.854731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277efa3-0d04-45dc-9b10-3c10db86b581 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:54.858755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.859572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.859859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.860485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.862139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:54.863640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:54.863640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:54.863640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] PATH_INFO: `/v3/projects/47872575b3ea451ca932797b667453cc` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:54.865387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47872575b3ea451ca932797b667453cc)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.867710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.872327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:54.874464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:54.884968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1068/2131] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:54.959660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47872575b3ea451ca932797b667453cc', 'request_id': 'req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.968252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47872575b3ea451ca932797b667453cc', 'request_id': 'req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.975587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47872575b3ea451ca932797b667453cc', 'request_id': 'req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.980407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:54.994607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] The token cache is being invalidate because project 47872575b3ea451ca932797b667453cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:54.995792 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:54.997081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0e27fd76-3fe7-402f-93ed-86863ad5a7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.001370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1064/2132] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:54 2022] DELETE /identity/v3/projects/47872575b3ea451ca932797b667453cc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:55.262010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.263587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:55.277717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da0b1-a6a3-4052-8610-715cb2e05c98 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.353897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.355227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.355227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.355537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] PATH_INFO: `/v3/projects/d6934da15a2046908e0d8c5907d8a76b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.359015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6934da15a2046908e0d8c5907d8a76b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.361583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.417352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6934da15a2046908e0d8c5907d8a76b', 'request_id': 'req-c59d386e-863d-4aca-87e4-a55e54e2af97'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.423471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6934da15a2046908e0d8c5907d8a76b', 'request_id': 'req-c59d386e-863d-4aca-87e4-a55e54e2af97'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.436488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6934da15a2046908e0d8c5907d8a76b', 'request_id': 'req-c59d386e-863d-4aca-87e4-a55e54e2af97'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.452994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] The token cache is being invalidate because project d6934da15a2046908e0d8c5907d8a76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:55.452994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.453865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c59d386e-863d-4aca-87e4-a55e54e2af97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:55.455865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1069/2133] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:55 2022] DELETE /identity/v3/projects/d6934da15a2046908e0d8c5907d8a76b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:55.465834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.466380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.466380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.490424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:55.491712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:55.566389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1065/2134] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:55.576036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.576651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.577016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.598208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:55.599773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:55.612181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1070/2135] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:55.616633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.628219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759e85b-81ef-4632-8cb4-2acde6df9813 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.638448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.639194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.639521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.639837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.640718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:55.641054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.643628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.660034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1066/2136] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:55.663845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.674531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f651499-42fa-4318-a0c1-8033e2983c02 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.679466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.680201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.680487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.680769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.681492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.683424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.687307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:55.723146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1071/2137] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:55.724538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.737154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a31aa-5877-41e2-90ce-64a4771223ff admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.810610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.811201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.811494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.811950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.812693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.814760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.837178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1067/2138] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:55.842392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.853480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719111be-2047-49df-9f8d-cd396c5d5ed2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.858117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.858822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.859097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.859370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.860171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.861841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.880318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1072/2139] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:55.884085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.894529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059323a8-cf7f-474e-ad93-7e317fba1499 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.899421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.900117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.900402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.900680 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.901440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.903038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd1905-e85f-4ed3-ae8c-b4a2b437b9ac admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.913996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1068/2140] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:55.917904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:55.928339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf576cce-744a-4da7-aec6-cf97a644b3b7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:55.932994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:55.933982 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.934353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.935135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] PATH_INFO: `/v3/projects/55da00320a4548938d34d0e1ff6e32d1/users/84f475d3523046eab9c2b77731f3920e/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:55.952730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55da00320a4548938d34d0e1ff6e32d1, user_id=84f475d3523046eab9c2b77731f3920e, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:55.955594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:55.969795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1073/2141] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:55 2022] PUT /identity/v3/projects/55da00320a4548938d34d0e1ff6e32d1/users/84f475d3523046eab9c2b77731f3920e/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:55.980408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:55.980580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:55.980890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.013426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] MFA Rules not processed for user `84f475d3523046eab9c2b77731f3920e`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:56.017726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.098551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.122138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1069/2142] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:56.133887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.136746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.196108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040269c9-e682-4a2d-8c47-888c27e44b17 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.199755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.200439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.200862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.201654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.211162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.212654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.222802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1074/2143] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:56.255217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.348677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a98fb8-5f42-4764-9847-d816e3f58457 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.363286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.363286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.363286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.363286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] PATH_INFO: `/v3/users/84f475d3523046eab9c2b77731f3920e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.363815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f475d3523046eab9c2b77731f3920e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.363815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.404978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.407566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.424365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac26421-7c17-4781-920a-108bf297fe3c service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.429622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.429974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f475d3523046eab9c2b77731f3920e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.430724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.431102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.431492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] PATH_INFO: `/v3/users/edec8c6a6ee84447825519b3000daac3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.432822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=edec8c6a6ee84447825519b3000daac3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.434846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.446098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f475d3523046eab9c2b77731f3920e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.448096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f475d3523046eab9c2b77731f3920e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.456555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f475d3523046eab9c2b77731f3920e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.466485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a0ec443-f418-4b1a-bd35-8793cbeecab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f475d3523046eab9c2b77731f3920e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.475474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1070/2144] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:56 2022] DELETE /identity/v3/users/84f475d3523046eab9c2b77731f3920e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:56.480701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a56f2-baea-4026-b99e-749682af6761 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.481369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a56f2-baea-4026-b99e-749682af6761 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.481963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a56f2-baea-4026-b99e-749682af6761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.502016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edec8c6a6ee84447825519b3000daac3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.523051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edec8c6a6ee84447825519b3000daac3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.523943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edec8c6a6ee84447825519b3000daac3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.532847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edec8c6a6ee84447825519b3000daac3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.539105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-994a56f2-baea-4026-b99e-749682af6761 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:56.544671 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-994a56f2-baea-4026-b99e-749682af6761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.545053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ed3414e8-abc2-4778-b6b8-ac0fd989ca3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edec8c6a6ee84447825519b3000daac3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:56.554481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1075/2145] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:56 2022] DELETE /identity/v3/users/edec8c6a6ee84447825519b3000daac3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:56.558889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.559274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.559648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.588825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:56.590170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.611291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.676942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1071/2146] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:56.679928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1076/2147] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:56.700581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a56f2-baea-4026-b99e-749682af6761 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.702128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:56.703687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-994a56f2-baea-4026-b99e-749682af6761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.704717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:56.716170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:56.787117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01430781-fa08-4e9d-9b94-f86cceb3ec17 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.790828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.791725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.791841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.792258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.803030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.804590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.808923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a56f2-baea-4026-b99e-749682af6761 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:56.814255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:56.814913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:56.815198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:56.815633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1077/2148] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:56.815752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:56.825848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:56.827409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:56.838213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1072/2149] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:57.285306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:57.286889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:57.300826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab417-d6a1-4e41-a3f9-9c5029899cd4 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:57.306692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:57.307605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.307990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.308191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] PATH_INFO: `/v3/projects/55da00320a4548938d34d0e1ff6e32d1` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.309467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55da00320a4548938d34d0e1ff6e32d1)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:57.311419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:57.361927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55da00320a4548938d34d0e1ff6e32d1', 'request_id': 'req-571b025d-e113-4ad7-ab69-619d29887b85'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.369846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55da00320a4548938d34d0e1ff6e32d1', 'request_id': 'req-571b025d-e113-4ad7-ab69-619d29887b85'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.380515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55da00320a4548938d34d0e1ff6e32d1', 'request_id': 'req-571b025d-e113-4ad7-ab69-619d29887b85'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.401565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] The token cache is being invalidate because project 55da00320a4548938d34d0e1ff6e32d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:57.401970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.402785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.404788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1078/2150] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:57 2022] DELETE /identity/v3/projects/55da00320a4548938d34d0e1ff6e32d1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:57.478800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:57.485505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:57.517591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799de529-9550-4fea-86e5-f1b38f345d0e service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:57.596277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:57.597503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.597934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.598354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] PATH_INFO: `/v3/projects/a64f28d4ec044d54b61672f9bb89efff` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.604015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64f28d4ec044d54b61672f9bb89efff)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:57.606785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:57.674881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64f28d4ec044d54b61672f9bb89efff', 'request_id': 'req-3a2da05d-2ed6-4798-9918-0761dd0020f1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.682529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64f28d4ec044d54b61672f9bb89efff', 'request_id': 'req-3a2da05d-2ed6-4798-9918-0761dd0020f1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.690041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64f28d4ec044d54b61672f9bb89efff', 'request_id': 'req-3a2da05d-2ed6-4798-9918-0761dd0020f1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.719749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] The token cache is being invalidate because project a64f28d4ec044d54b61672f9bb89efff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:57.720094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.720817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3a2da05d-2ed6-4798-9918-0761dd0020f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.722828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1073/2151] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:57 2022] DELETE /identity/v3/projects/a64f28d4ec044d54b61672f9bb89efff => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:57.726737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:57.728338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:57.746885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b025d-e113-4ad7-ab69-619d29887b85 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:57.810957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:57.827353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:57.842799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:57.843150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:57.843529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] PATH_INFO: `/v3/users/94cebc244c0441f08c277002916b55a3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:57.845295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94cebc244c0441f08c277002916b55a3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:57.847753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:57.918124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94cebc244c0441f08c277002916b55a3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.939118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94cebc244c0441f08c277002916b55a3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.940004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94cebc244c0441f08c277002916b55a3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.950006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94cebc244c0441f08c277002916b55a3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.961324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ac495608-e448-43da-8db6-f55a094fdc24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94cebc244c0441f08c277002916b55a3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:57.971052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1079/2152] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:57 2022] DELETE /identity/v3/users/94cebc244c0441f08c277002916b55a3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:58.355307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.355697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.356237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.357514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1074/2153] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:57:58.395980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.396550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.396824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.415676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] MFA Rules not processed for user `17f4edebe2ed45a08ef7460d812c563d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:58.417085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.460616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.462943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.534381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1080/2154] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:58.534538 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.535765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.537460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.560961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8e064-7bc3-48ea-af2d-7bda3df61ca0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.566237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.567134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.567416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.567689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] PATH_INFO: `/v3/users/5cc60574eb7f4df6b193c244973ed8c3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.568930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc60574eb7f4df6b193c244973ed8c3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.570849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.618346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc60574eb7f4df6b193c244973ed8c3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.625623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac4564-bb51-464c-8116-bb2fb3228029 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.630137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.630974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc60574eb7f4df6b193c244973ed8c3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.631103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.631269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.631538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] PATH_INFO: `/v3/projects/09e220f7e6e6415e92478874ed8e7421` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.631754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc60574eb7f4df6b193c244973ed8c3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.632741 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e220f7e6e6415e92478874ed8e7421)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.634694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.638670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc60574eb7f4df6b193c244973ed8c3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.643402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.647782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc60574eb7f4df6b193c244973ed8c3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.655954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1075/2155] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:57:58 2022] DELETE /identity/v3/users/5cc60574eb7f4df6b193c244973ed8c3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:58.657118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.658593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.695717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e220f7e6e6415e92478874ed8e7421', 'request_id': 'req-005de6a2-86d6-4bce-9a17-8e9905c5682e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.701703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e220f7e6e6415e92478874ed8e7421', 'request_id': 'req-005de6a2-86d6-4bce-9a17-8e9905c5682e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.710939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e220f7e6e6415e92478874ed8e7421', 'request_id': 'req-005de6a2-86d6-4bce-9a17-8e9905c5682e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.740955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] The token cache is being invalidate because project 09e220f7e6e6415e92478874ed8e7421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:58.741552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.742531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-005de6a2-86d6-4bce-9a17-8e9905c5682e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:58.744337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1081/2156] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:58 2022] DELETE /identity/v3/projects/09e220f7e6e6415e92478874ed8e7421 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:58.750485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e69a9-77ac-4dc2-9c86-d9a041dda1f5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.750916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e69a9-77ac-4dc2-9c86-d9a041dda1f5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.751564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e69a9-77ac-4dc2-9c86-d9a041dda1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.765297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb1a77-1e93-4dbf-a976-65fea111996a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.784240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7d1e69a9-77ac-4dc2-9c86-d9a041dda1f5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:58.786268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d1e69a9-77ac-4dc2-9c86-d9a041dda1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.838454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.839540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.839915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.840270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.843045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.854355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:58.861880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.864517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.869574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1082/2157] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:58.880367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.880661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.880942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.903961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1076/2158] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:57:58.905451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.906945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:58.908115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.908296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:58.926308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1083/2159] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:58.930420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:58.933764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25f790-4157-4fc5-bea6-a9e1ab5b9fdd service placement] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.939635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.940907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.940907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.941123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] PATH_INFO: `/v3/projects/5dcd3b32ea75463c95bb2d451c59cdd1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.942216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dcd3b32ea75463c95bb2d451c59cdd1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.944741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.948530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822d61-3ddb-443d-aa29-af743ddd7289 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:58.964490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:58.965191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:58.965474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:58.965779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:58.966527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:58.966833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:58.975130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:58.982657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1084/2160] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:57:58.987258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.000130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42352204-f9c3-4c91-a16f-0260a5c84ab7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.008113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.008855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.009693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.009693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.010053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.011935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.018431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dcd3b32ea75463c95bb2d451c59cdd1', 'request_id': 'req-7799f987-3c67-4a31-abc1-689a9bba468e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:59.024797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dcd3b32ea75463c95bb2d451c59cdd1', 'request_id': 'req-7799f987-3c67-4a31-abc1-689a9bba468e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:59.031105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dcd3b32ea75463c95bb2d451c59cdd1', 'request_id': 'req-7799f987-3c67-4a31-abc1-689a9bba468e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:59.039433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1085/2161] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:58 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:59.043742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.048231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] The token cache is being invalidate because project 5dcd3b32ea75463c95bb2d451c59cdd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:57:59.048655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:59.049315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7799f987-3c67-4a31-abc1-689a9bba468e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:57:59.050886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1077/2162] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:57:58 2022] DELETE /identity/v3/projects/5dcd3b32ea75463c95bb2d451c59cdd1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:57:59.055909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42d1fa-b481-4cc9-a992-7954df2305df None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.056223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42d1fa-b481-4cc9-a992-7954df2305df None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.056506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42d1fa-b481-4cc9-a992-7954df2305df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.075113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ad42d1fa-b481-4cc9-a992-7954df2305df None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.076373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ad42d1fa-b481-4cc9-a992-7954df2305df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.101694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.108425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1631-ff00-4fb7-9141-88dc9696554b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.115907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.116536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.117118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.117596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.119013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.121449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.140769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1078/2163] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:59.144917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1086/2164] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:57:59.150987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.150987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.150987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.152248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.154574 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.169420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.170639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.171283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6050-34e8-470a-b0de-72467d67530b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.177622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.178420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.178713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.178987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.179800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1079/2165] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:59.180002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.181914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.183675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.193107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c0a1-d44c-443a-9140-77e78cf6b8c8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.196475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1087/2166] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:59.197367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.200653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.201320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.201604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.201880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.202707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:57:59.203019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.204689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.207413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473074e-3d48-483f-ae3c-9c5ee4760489 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.212183 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.212868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.213153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.213434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.214206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.215839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.229457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1080/2167] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:57:59.230825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.243258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1088/2168] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:59.245176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d500-8c8a-4102-8180-caf3f939236b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.247537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.250743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.251430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.251716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.252041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.252754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.254715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.259995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbdd6e-17a5-4567-8cf3-65c0ea490ac4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.265419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.266338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.266621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.266898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] PATH_INFO: `/v3/projects/b933118abf2e4260a5d1b78b5a2ae452/users/9b963031ee5a45e48b38cbb912535b2e/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.285253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b933118abf2e4260a5d1b78b5a2ae452, user_id=9b963031ee5a45e48b38cbb912535b2e, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.288023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4f025-eade-41a4-9729-3d08ebe0c205 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.288499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1081/2169] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:59.292954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.308066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1089/2170] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:59 2022] PUT /identity/v3/projects/b933118abf2e4260a5d1b78b5a2ae452/users/9b963031ee5a45e48b38cbb912535b2e/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:59.320934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.321414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.321701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.337518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.353095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] MFA Rules not processed for user `9b963031ee5a45e48b38cbb912535b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.354416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.366169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be091a6f-e541-4d1f-8f2d-21af887d4e67 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.431515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1090/2171] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:57:59.431773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.431773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.431773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.431773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.432147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.433290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.439035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.440473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.455978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1082/2172] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:59.460857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.478518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deaef-b0d0-484a-98b8-76de98f7ce45 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.486447 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.487743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.488484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.488843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.489702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.491903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.508492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1083/2173] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:57:59.510520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.522381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f0186-8e1f-420e-81ce-18ebb9282b83 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.527219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.528016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.528364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.528701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.529561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.531377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.546670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1084/2174] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:57:59.547802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.553677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989eebd5-5e55-450c-b9c9-6892ae7bde7d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.557443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.558162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.558506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.558844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.566309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf029-64c4-431e-94c6-ccd75d8f7653 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.567537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.569033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5af9f2-6330-4f1a-95ec-e88ba0033979 service glance] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.571120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.572072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.572355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.572629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] PATH_INFO: `/v3/projects/0f1eaf5343e54a709b00d86c9bc60108/users/e25cddd13ea547c0bbb9401fbed55e55/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.578256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1091/2175] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:57:59.587623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1eaf5343e54a709b00d86c9bc60108, user_id=e25cddd13ea547c0bbb9401fbed55e55, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.590309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.604102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1085/2176] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:59 2022] PUT /identity/v3/projects/0f1eaf5343e54a709b00d86c9bc60108/users/e25cddd13ea547c0bbb9401fbed55e55/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:57:59.614241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.614552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.614835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.635073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.644686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] MFA Rules not processed for user `e25cddd13ea547c0bbb9401fbed55e55`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:57:59.646134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:57:59.719772 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1092/2177] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:57:59.724842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.793551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae163f3-37ce-418d-95d2-1177ae9f7e59 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.798325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.798980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.799259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.799530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.800324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.802217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.834477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1086/2178] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:59.838659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.849358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644898d-2018-4a48-82c1-c141845d2597 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.910908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.911590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.911890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.912604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.914381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.915493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.944066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1093/2179] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:57:59 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:57:59.948364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:57:59.959358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828ccb6-f97f-488f-9b07-2a6a8dff9eee admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:57:59.964262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:57:59.964888 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:57:59.965173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:57:59.965455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:57:59.966181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:57:59.967866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:57:59.973718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:57:59.990886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.003668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1087/2180] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:00.011691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ef503-cec9-4d8a-ac04-084f35e82c6c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.017759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.019002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.019290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.019570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] PATH_INFO: `/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb/users/f398b99927dc4e2f84b557a7b8c86440/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.038727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e498bf31c4a6415d8f1fb5b4d3f9edcb, user_id=f398b99927dc4e2f84b557a7b8c86440, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.041623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.060507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1094/2181] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:57:59 2022] PUT /identity/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb/users/f398b99927dc4e2f84b557a7b8c86440/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:00.065154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.075741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41f1-216a-4b8d-9541-84354c7075f0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.146785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.147459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.147739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.148054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.148785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.150407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4c0d3-39f5-45ec-aa2c-25772378bad0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.160897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1088/2182] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:58:00.165131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.176001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18213-22b6-49b3-9dac-69f1b3a2bfb8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.180740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.181620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.181900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.182174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] PATH_INFO: `/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb/users/f398b99927dc4e2f84b557a7b8c86440/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.184181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e498bf31c4a6415d8f1fb5b4d3f9edcb, user_id=f398b99927dc4e2f84b557a7b8c86440, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.186838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.197316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1095/2183] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:00 2022] PUT /identity/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb/users/f398b99927dc4e2f84b557a7b8c86440/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:00.207667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.208015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.208308 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.239835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] MFA Rules not processed for user `f398b99927dc4e2f84b557a7b8c86440`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:00.241109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:00.316328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1089/2184] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:00.328451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.330218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:00.404474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d7b13-8e0a-439d-98cb-70eb99665e85 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.409441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.410160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.410537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.410872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.419031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:00.422793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.425154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.436333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1096/2185] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:00.489957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:00.493421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:00.510010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27640bbf-ac41-4186-ab23-54c3830f628c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:00.514828 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:00.515463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:00.515732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:00.516033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:00.583031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:00.584537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1359-f344-4b5b-b89b-630619d01e40 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:00.595459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1090/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:03.973946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.043032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86dd0-6c25-4d0d-bde9-dc7e081a6371 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.048276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.049157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.049436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.049716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] PATH_INFO: `/v3/users/e25cddd13ea547c0bbb9401fbed55e55` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.050981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25cddd13ea547c0bbb9401fbed55e55)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.053088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.103763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25cddd13ea547c0bbb9401fbed55e55'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.116865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25cddd13ea547c0bbb9401fbed55e55'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.117605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25cddd13ea547c0bbb9401fbed55e55'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.124987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25cddd13ea547c0bbb9401fbed55e55'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.135601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25cddd13ea547c0bbb9401fbed55e55'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.144976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1097/2187] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:03 2022] DELETE /identity/v3/users/e25cddd13ea547c0bbb9401fbed55e55 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:04.150070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.150608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.150850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.170002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:04.171837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:04.185408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:04.252137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1091/2188] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:58:04.259035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.260958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:04.323660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29286c-7661-4367-87e2-fad5e543ea4a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.327785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.328415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.328701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.328976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.338809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.340283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.350109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1098/2189] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:04.823757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.825373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:04.854001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477932c-c0b3-4ef6-b7b1-82c3bc9e2574 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.859696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:04.860614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:04.860897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:04.861371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] PATH_INFO: `/v3/projects/0f1eaf5343e54a709b00d86c9bc60108` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:04.862529 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1eaf5343e54a709b00d86c9bc60108)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:04.864510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:04.916576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1eaf5343e54a709b00d86c9bc60108', 'request_id': 'req-985ac63f-5a8b-4205-9268-f7cf1fa281d8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.923443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1eaf5343e54a709b00d86c9bc60108', 'request_id': 'req-985ac63f-5a8b-4205-9268-f7cf1fa281d8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.929112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1eaf5343e54a709b00d86c9bc60108', 'request_id': 'req-985ac63f-5a8b-4205-9268-f7cf1fa281d8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.943648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] The token cache is being invalidate because project 0f1eaf5343e54a709b00d86c9bc60108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:04.943955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.944668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:04.946327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1092/2190] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:04 2022] DELETE /identity/v3/projects/0f1eaf5343e54a709b00d86c9bc60108 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:04.950447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:04.960382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee593fc0-fd5b-493c-9bf4-ce9729ce4d0a service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:04.974277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:05.028736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.029959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.030355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.030707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] PATH_INFO: `/v3/users/f398b99927dc4e2f84b557a7b8c86440` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.032534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f398b99927dc4e2f84b557a7b8c86440)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:05.034501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:05.084362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f398b99927dc4e2f84b557a7b8c86440'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.098393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f398b99927dc4e2f84b557a7b8c86440'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.099166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f398b99927dc4e2f84b557a7b8c86440'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.110219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f398b99927dc4e2f84b557a7b8c86440'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.119898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4f7e710-e1d2-403e-a8e2-81642b979efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f398b99927dc4e2f84b557a7b8c86440'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.126952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1099/2191] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:04 2022] DELETE /identity/v3/users/f398b99927dc4e2f84b557a7b8c86440 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:05.479712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.481774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:05.503429 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ac63f-5a8b-4205-9268-f7cf1fa281d8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.569264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.570139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.570420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.570690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] PATH_INFO: `/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.571976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e498bf31c4a6415d8f1fb5b4d3f9edcb)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:05.573971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:05.621778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:05.625541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e498bf31c4a6415d8f1fb5b4d3f9edcb', 'request_id': 'req-a4d10e83-ca87-4863-8458-943b75cf9141'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.633150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e498bf31c4a6415d8f1fb5b4d3f9edcb', 'request_id': 'req-a4d10e83-ca87-4863-8458-943b75cf9141'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.638521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e498bf31c4a6415d8f1fb5b4d3f9edcb', 'request_id': 'req-a4d10e83-ca87-4863-8458-943b75cf9141'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.657210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] The token cache is being invalidate because project e498bf31c4a6415d8f1fb5b4d3f9edcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:05.657853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.660966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a4d10e83-ca87-4863-8458-943b75cf9141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:05.664048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1093/2192] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:05 2022] DELETE /identity/v3/projects/e498bf31c4a6415d8f1fb5b4d3f9edcb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:05.670083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.670380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.670683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.690835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:05.692063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:05.758960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1100/2193] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:05.767393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161094-97e4-4594-a279-78d14c659490 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.767687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161094-97e4-4594-a279-78d14c659490 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.768047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161094-97e4-4594-a279-78d14c659490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.792760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-67161094-97e4-4594-a279-78d14c659490 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:05.793961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-67161094-97e4-4594-a279-78d14c659490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:05.803231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1094/2194] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:05.807863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.817972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c81e5-69b4-49cb-8e44-d67e2d0af16c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.823173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.823832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.824140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.824414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.825176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:05.825481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:05.827091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:05.839852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1101/2195] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:05.846182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67161094-97e4-4594-a279-78d14c659490 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.858780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67161094-97e4-4594-a279-78d14c659490 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.881547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:05.882233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:05.882544 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:05.882795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:05.883519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:05.885608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:05.919028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1095/2196] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:58:05.924278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:05.934917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26365f96-584d-44df-831b-4fd260137828 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:05.964760 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.015645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.016396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.016666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.016943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.017649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.019949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.045295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1102/2197] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:05 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:06.049631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.059941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753fd7f-08c8-4b93-9bdb-e749bde93a42 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.064599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.065259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.065545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.065822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.066551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.068231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.079177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1096/2198] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:06.083330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.093334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da2c8-70ce-4e1e-8b9d-2e18c42bfe62 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.097987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.098639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.099118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.099448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.100432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.102667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf0b17-ec27-4c86-8cd3-756c601343ae admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.115069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1103/2199] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:58:06.119451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.129931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39e10b-123e-4d69-bd46-3979441a3a42 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.134912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.135789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.136171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.136440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] PATH_INFO: `/v3/projects/4ba86269d6bb4cc7b92040315c374e23/users/69def43ea43342929d8ed05c5e38974c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.153971 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba86269d6bb4cc7b92040315c374e23, user_id=69def43ea43342929d8ed05c5e38974c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.159343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.177563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1097/2200] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:06 2022] PUT /identity/v3/projects/4ba86269d6bb4cc7b92040315c374e23/users/69def43ea43342929d8ed05c5e38974c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:06.191977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.192272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.192552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.223415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] MFA Rules not processed for user `69def43ea43342929d8ed05c5e38974c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:06.224669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:06.294170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1104/2201] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:06.301514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.372347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159468a-0152-4548-a0fb-01447666009e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.378421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.379129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.379409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.379650 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.380414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.382330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.410660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1098/2202] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:06.414933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.425863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de885e-b6a9-4e47-a169-2a143cdc2d70 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.490564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.491453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.491863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.492289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.493333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.496407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.521906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1105/2203] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:06.524827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.535380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd99f3-d096-459e-92e0-bb8459380e19 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.540353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.541005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.541254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.541524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.542245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.543826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.555052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1099/2204] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:06.559920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.576167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1167176-4a7d-4547-9a0f-37396993cd36 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.582542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.583447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.583747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.584056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] PATH_INFO: `/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf/users/550abd8b38e74d518469008856801bb5/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.606209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a524eeaa36fb4c61bb96439e37be3fcf, user_id=550abd8b38e74d518469008856801bb5, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.609173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.619203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.626201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1106/2205] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:06 2022] PUT /identity/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf/users/550abd8b38e74d518469008856801bb5/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:06.630343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.640583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f816d-6c8f-4268-8960-a2bf5cc4b76d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.687770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:06.704686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.705358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.705640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.705916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.706678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.708350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43797ac8-f98f-4ff1-a953-31a172d84d36 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.718678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1100/2206] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:06.723100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.733987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53230b-a908-4691-8769-36cfa19be3bf admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.738962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.739989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.740413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.740687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] PATH_INFO: `/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf/users/550abd8b38e74d518469008856801bb5/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.742777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a524eeaa36fb4c61bb96439e37be3fcf, user_id=550abd8b38e74d518469008856801bb5, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.745507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.756315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1107/2207] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:06 2022] PUT /identity/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf/users/550abd8b38e74d518469008856801bb5/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:06.767757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.768084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.768685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.799381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] MFA Rules not processed for user `550abd8b38e74d518469008856801bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:06.800696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:06.876125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1101/2208] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:06.887805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:06.889895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:06.955975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149aee2-7469-4d1f-a6a0-77dbb98112e4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:06.962680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:06.963417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:06.963641 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:06.963914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:06.974335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:06.975781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:06.985589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1108/2209] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:07.075228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:07.077302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:07.090417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcdf87-41a1-4e70-a71e-a4d6ad8a8301 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:07.094442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:07.095118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:07.095402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:07.095662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:07.154478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:07.155926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee622bbd-dfdb-4355-ab04-1dbc0c8b834b service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:07.170224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1102/2210] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:12.057052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.119166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ad35a-7fc8-496a-b071-679bb729bb49 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.124039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.125173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.125446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.125720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] PATH_INFO: `/v3/users/69def43ea43342929d8ed05c5e38974c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.126960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69def43ea43342929d8ed05c5e38974c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:12.128957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:12.184646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69def43ea43342929d8ed05c5e38974c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.197093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69def43ea43342929d8ed05c5e38974c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.197780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69def43ea43342929d8ed05c5e38974c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.205346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69def43ea43342929d8ed05c5e38974c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.214108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69def43ea43342929d8ed05c5e38974c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.220705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1109/2211] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:12 2022] DELETE /identity/v3/users/69def43ea43342929d8ed05c5e38974c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:12.227391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.227929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.228042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.247779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:12.249032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.316027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1103/2212] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:12.329001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.330692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.341759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.400062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae57647-e892-4433-964b-0bd7abf3e867 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.404228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.404878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.405160 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.405459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.415277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:12.416780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:12.427317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1110/2213] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:12.819411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.821034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:12.839858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:12.852542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b38d2-c28a-4623-a625-8fe6f29f4f32 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:12.857968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:12.859600 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:12.859923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:12.860201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] PATH_INFO: `/v3/projects/4ba86269d6bb4cc7b92040315c374e23` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:12.861684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba86269d6bb4cc7b92040315c374e23)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:12.863642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:12.915182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba86269d6bb4cc7b92040315c374e23', 'request_id': 'req-bcd1ea03-c28f-4828-9fe3-daaa92881101'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.922554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba86269d6bb4cc7b92040315c374e23', 'request_id': 'req-bcd1ea03-c28f-4828-9fe3-daaa92881101'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.928947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba86269d6bb4cc7b92040315c374e23', 'request_id': 'req-bcd1ea03-c28f-4828-9fe3-daaa92881101'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.943752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] The token cache is being invalidate because project 4ba86269d6bb4cc7b92040315c374e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:12.944109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.944773 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:12.946448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1104/2214] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:12 2022] DELETE /identity/v3/projects/4ba86269d6bb4cc7b92040315c374e23 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:12.950727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:12.961516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f836c5-2d7d-45ac-b982-25c10937231e service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.023841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.024746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.025034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.025307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] PATH_INFO: `/v3/users/550abd8b38e74d518469008856801bb5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.026502 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550abd8b38e74d518469008856801bb5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.028498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.083740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550abd8b38e74d518469008856801bb5'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.096535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550abd8b38e74d518469008856801bb5'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.097230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550abd8b38e74d518469008856801bb5'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.104334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550abd8b38e74d518469008856801bb5'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.113951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f5a67f9c-a3c2-46bc-9a07-fe753a0c6069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550abd8b38e74d518469008856801bb5'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.120067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1111/2215] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:12 2022] DELETE /identity/v3/users/550abd8b38e74d518469008856801bb5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:13.414014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.415595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:13.430865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1ea03-c28f-4828-9fe3-daaa92881101 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.490377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.491170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.491443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.491709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] PATH_INFO: `/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.492917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a524eeaa36fb4c61bb96439e37be3fcf)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.494836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.548484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a524eeaa36fb4c61bb96439e37be3fcf', 'request_id': 'req-87dc8405-9a8b-4819-89a3-6b82cab41ab4'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.555074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a524eeaa36fb4c61bb96439e37be3fcf', 'request_id': 'req-87dc8405-9a8b-4819-89a3-6b82cab41ab4'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.561744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a524eeaa36fb4c61bb96439e37be3fcf', 'request_id': 'req-87dc8405-9a8b-4819-89a3-6b82cab41ab4'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.576521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] The token cache is being invalidate because project a524eeaa36fb4c61bb96439e37be3fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:13.576818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.577516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-87dc8405-9a8b-4819-89a3-6b82cab41ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:13.579058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1105/2216] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:13 2022] DELETE /identity/v3/projects/a524eeaa36fb4c61bb96439e37be3fcf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:13.583985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.584272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.584548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.604250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:13.605482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:13.630912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.670357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1112/2217] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:13.678774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.679071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.679351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.697988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:13.699193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:13.708352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1106/2218] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:13.712866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.723104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a2e5b-00a3-48c1-ad05-d2c9487dad87 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.728509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.729174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.729454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.729724 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.730449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:13.730748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.732753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.752535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1113/2219] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:13.756976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.767438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb67d4a-94b2-4645-9c3a-c29fa730eeab None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.772578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.773241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.773572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.773848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.774539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.776506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.792664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:13.806704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1107/2220] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:13 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:13.810884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.825902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41ca9-f33a-4dfd-b015-256362bf73f4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.884086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.884770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.885056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.885331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.886037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.888008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.912171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1114/2221] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:13 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:58:13.913237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.926576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63b1a-4aba-4cc7-8f2e-97e117cbf855 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.932126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.932800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.933081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.933357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.934113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.935801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.946744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1108/2222] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:13.950806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.961293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a752-5026-44b4-89d7-2ac3b436e44d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:13.966258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:13.966912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:13.967188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:13.967465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:13.968269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:13.969861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f39d0-e61b-4d67-8bdc-79676e0e6e15 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:13.980002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1115/2223] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:13.984566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:13.997105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129f73f-3f05-4d84-ba1b-0e591f7ea292 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.002386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.003351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.003564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.003839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] PATH_INFO: `/v3/projects/fbd0a0b81ee94640a98a4c95bf5886e3/users/39d4ecb56a1c48f5ad30ba80e61824c2/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.018488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd0a0b81ee94640a98a4c95bf5886e3, user_id=39d4ecb56a1c48f5ad30ba80e61824c2, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.021525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.036543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1109/2224] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:13 2022] PUT /identity/v3/projects/fbd0a0b81ee94640a98a4c95bf5886e3/users/39d4ecb56a1c48f5ad30ba80e61824c2/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:14.048141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.048538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.048933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.089186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] MFA Rules not processed for user `39d4ecb56a1c48f5ad30ba80e61824c2`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:14.090510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:14.160417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1116/2225] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:14.164896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.228886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12467a7b-507e-465f-b217-48039d9c2380 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.234563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.235202 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.235485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.235762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.236484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.238390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.273779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1110/2226] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:14.278585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.289359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66de13b-e21f-47a3-8d81-0951a84a7d8c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.311539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.354243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.354919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.355199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.355474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.356197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.358159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.380051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1117/2227] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:14.384224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.395238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33694-e983-4a82-ac21-c8f59129a54c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.401413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.402179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.402503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.402961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.404647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.406238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.417856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1111/2228] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:14.422394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.432931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757a7ea-1902-46cb-8a83-cb5a10876039 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.438182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.439545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.439836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.440147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] PATH_INFO: `/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808/users/9b90a2ade47a41d9b56000c140131501/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.456110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c1c9ab2c9c4c4ab724344ba50f0808, user_id=9b90a2ade47a41d9b56000c140131501, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.458852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.474483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1118/2229] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:14 2022] PUT /identity/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808/users/9b90a2ade47a41d9b56000c140131501/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:14.479548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.490467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94c3b-71aa-4f9d-b0ef-0dfa7bcca54e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.550843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.551741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.551980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.552255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.552990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.554862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58be6-fd92-4984-8cd2-b8d44e3b94f1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.573179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1112/2230] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:14.577465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.595217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acd79b-524d-483d-b387-5d42b9448cd6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.595217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.595217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.595217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.595529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] PATH_INFO: `/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808/users/9b90a2ade47a41d9b56000c140131501/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.596487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c1c9ab2c9c4c4ab724344ba50f0808, user_id=9b90a2ade47a41d9b56000c140131501, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.599109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.610527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1119/2231] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:14 2022] PUT /identity/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808/users/9b90a2ade47a41d9b56000c140131501/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:14.621646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.621970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.622277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.653862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] MFA Rules not processed for user `9b90a2ade47a41d9b56000c140131501`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:14.655180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:14.667440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:14.736591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1113/2232] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:14.748778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.750775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:14.817985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbea35-7d75-4eff-bc64-c77642124890 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.821840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.822500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.822788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.823068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:14.833211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:14.834653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:14.845025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1120/2233] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:58:14.963585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:14.979175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430cffe-fab1-4e9b-8d67-a7f64737337e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:14.983251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:14.983983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:14.984271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:14.984551 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.064427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.065889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599d8ff-50f2-4913-9e87-2260d2492360 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.075091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1114/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:15.360148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.430521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75871bf-8aed-4b58-a61b-ed63cb55f3b8 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.435611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.436626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.437093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.437377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] PATH_INFO: `/v3/users/39d4ecb56a1c48f5ad30ba80e61824c2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.438672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d4ecb56a1c48f5ad30ba80e61824c2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.440892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.466319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:15.495943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d4ecb56a1c48f5ad30ba80e61824c2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:15.510872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d4ecb56a1c48f5ad30ba80e61824c2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:15.511631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d4ecb56a1c48f5ad30ba80e61824c2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:15.519591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d4ecb56a1c48f5ad30ba80e61824c2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:15.535504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d4ecb56a1c48f5ad30ba80e61824c2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:15.549419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1121/2235] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:15 2022] DELETE /identity/v3/users/39d4ecb56a1c48f5ad30ba80e61824c2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:15.554122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.554719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.554719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.573201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:15.574403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:15.647664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1115/2236] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:15.657573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:15.659308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:15.719111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6abb2-91d0-4aa6-a1b5-2fd38bd7379d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:15.723107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:15.723742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:15.724052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:15.724349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:15.733700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:15.735271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:15.745232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1122/2237] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:16.030059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.031927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:16.044862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d351-3d58-45ee-a19b-2c7616b0da75 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.051352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.052408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.052923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.053303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] PATH_INFO: `/v3/projects/fbd0a0b81ee94640a98a4c95bf5886e3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.055683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd0a0b81ee94640a98a4c95bf5886e3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.065042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.113304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.117586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd0a0b81ee94640a98a4c95bf5886e3', 'request_id': 'req-1a726523-f027-42ec-9be5-a890d2f62040'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.125025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd0a0b81ee94640a98a4c95bf5886e3', 'request_id': 'req-1a726523-f027-42ec-9be5-a890d2f62040'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.131127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd0a0b81ee94640a98a4c95bf5886e3', 'request_id': 'req-1a726523-f027-42ec-9be5-a890d2f62040'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.146339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] The token cache is being invalidate because project fbd0a0b81ee94640a98a4c95bf5886e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:16.146639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.147264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.148839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1116/2238] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:16 2022] DELETE /identity/v3/projects/fbd0a0b81ee94640a98a4c95bf5886e3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:16.152980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.164983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a086b6c-357f-47d4-b56e-0ccb4c30bf4d service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.230448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.231336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.231620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.231909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] PATH_INFO: `/v3/users/9b90a2ade47a41d9b56000c140131501` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.233154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b90a2ade47a41d9b56000c140131501)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.235189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.289570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:16.292365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b90a2ade47a41d9b56000c140131501'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.307318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b90a2ade47a41d9b56000c140131501'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.308075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b90a2ade47a41d9b56000c140131501'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.315504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b90a2ade47a41d9b56000c140131501'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.324052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-715d36f5-3969-4c81-95e8-a37d3de3f035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b90a2ade47a41d9b56000c140131501'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.330375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1123/2239] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:16 2022] DELETE /identity/v3/users/9b90a2ade47a41d9b56000c140131501 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:16.616198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.617795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:16.632194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a726523-f027-42ec-9be5-a890d2f62040 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.723654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.724562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.724831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.725105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] PATH_INFO: `/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.726825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c1c9ab2c9c4c4ab724344ba50f0808)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.729010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.780951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c1c9ab2c9c4c4ab724344ba50f0808', 'request_id': 'req-f757b3c8-e9c6-478e-853e-b9c475a012e9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.787049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c1c9ab2c9c4c4ab724344ba50f0808', 'request_id': 'req-f757b3c8-e9c6-478e-853e-b9c475a012e9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.792421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c1c9ab2c9c4c4ab724344ba50f0808', 'request_id': 'req-f757b3c8-e9c6-478e-853e-b9c475a012e9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.805675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] The token cache is being invalidate because project b1c1c9ab2c9c4c4ab724344ba50f0808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:16.806204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.807032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-f757b3c8-e9c6-478e-853e-b9c475a012e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:16.808661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1117/2240] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:16 2022] DELETE /identity/v3/projects/b1c1c9ab2c9c4c4ab724344ba50f0808 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:16.814027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.814561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.814670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.833430 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:16.834659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:16.907499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1124/2241] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:16.917260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.917260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.917260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.940164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:16.941668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:16.950872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1118/2242] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:16.955469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:16.965331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2b2d-07ba-4a67-b7a5-b63e9cb6334c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:16.970385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:16.971056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:16.971339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:16.971615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:16.972383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:16.972698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:16.974345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:16.988246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1125/2243] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:16.991116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.001825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbe4e-15ec-4743-a7fd-b4a969361b9c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.006846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.007512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.007837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.008147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.008859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.010981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.044617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1119/2244] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:16 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:17.046233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.056495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05256a-233e-4c79-a853-18bc57901114 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.117005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.117682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.117967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.118243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.118950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.120921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.140673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1126/2245] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:17 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:17.144983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.155320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3680b34-cf11-4b7c-a328-12a0b855618a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.160413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.161017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.161302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.161583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.162348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.164025 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.179473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1120/2246] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:17.180884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.191723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29825-b91a-4d2b-93d2-b5135086f0f7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.196824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.197489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.197765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.198036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.198809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.200452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee2772-caaa-4483-8e43-606cdd654f74 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.210921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1127/2247] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:17.215096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.226162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a91d11-f4a1-487a-a2cf-86620a86c7c5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.231728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.232777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.232950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.233232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] PATH_INFO: `/v3/projects/8a913265f84c43579ce0fb5a2604b663/users/a7d7f7af47db49d9a30db75c76980cb9/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.252041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a913265f84c43579ce0fb5a2604b663, user_id=a7d7f7af47db49d9a30db75c76980cb9, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.252041 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.260720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:17.269989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1121/2248] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:17 2022] PUT /identity/v3/projects/8a913265f84c43579ce0fb5a2604b663/users/a7d7f7af47db49d9a30db75c76980cb9/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:17.275796 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef68ab-c2aa-45d2-8f1e-6254a0c8857e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.276687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef68ab-c2aa-45d2-8f1e-6254a0c8857e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.276977 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef68ab-c2aa-45d2-8f1e-6254a0c8857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.298017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-96ef68ab-c2aa-45d2-8f1e-6254a0c8857e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:17.299460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-96ef68ab-c2aa-45d2-8f1e-6254a0c8857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:17.321400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:17.368966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1128/2249] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:58:17.378859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:17.380736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:17.439805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2cbe6-dbd3-45cb-8512-a0c079e43dab admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:17.443842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:17.444526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:17.444813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:17.445091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:17.455066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:17.456756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:17.466610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1122/2250] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:18.119985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.120539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.120690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.149337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] MFA Rules not processed for user `a7d7f7af47db49d9a30db75c76980cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:18.150717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:18.247889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1129/2251] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:18.258825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:18.260355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:18.330185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6118c-57bf-476e-aa31-56cdddb951de service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:18.333934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:18.334582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:18.334860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:18.335133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:18.345352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:18.346797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97488cca-79fd-4abe-88e2-020aab659d29 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:18.355573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1123/2252] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:19.533400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.536100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.554891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cb586-cbc4-4d8c-9480-aca466a543db None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.560551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.561402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.561679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.562092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] PATH_INFO: `/v3/users/d75355d82a89414da65c0a18ec637dd9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.563334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75355d82a89414da65c0a18ec637dd9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.565310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.615458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75355d82a89414da65c0a18ec637dd9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:19.627218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75355d82a89414da65c0a18ec637dd9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:19.627943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75355d82a89414da65c0a18ec637dd9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:19.634923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75355d82a89414da65c0a18ec637dd9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:19.644080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-67c57567-66e2-465b-9d73-bbfe429b3227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75355d82a89414da65c0a18ec637dd9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:19.650032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1130/2253] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:19 2022] DELETE /identity/v3/users/d75355d82a89414da65c0a18ec637dd9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:19.654782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.655396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.655396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.673289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:19.674522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.715963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.716407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.716681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.735360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] MFA Rules not processed for user `a7d7f7af47db49d9a30db75c76980cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:19.736572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1124/2254] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:19.736921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.747287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:19.748642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:19.761610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.795296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1131/2255] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:58:19.797633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:19.808850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61e4e5-57db-41e9-a40d-6dd1334f250d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:19.812388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:19.813032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:19.813311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:19.813580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:19.823267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:19.824298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c18a90-9620-424e-afd2-f483f18880ba service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:19.834256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1125/2256] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:58:20.127074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.129916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.160206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae8e0-d884-4e9f-89a3-6fe6031bbe5f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.167700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.168765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.169143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.169517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] PATH_INFO: `/v3/projects/f2da94a14ca94e13b067ae9fd79bbdac` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.170929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2da94a14ca94e13b067ae9fd79bbdac)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.173097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.222164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2da94a14ca94e13b067ae9fd79bbdac', 'request_id': 'req-2a9a327b-eed4-48fe-a371-29f2f14cb83c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:20.227714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2da94a14ca94e13b067ae9fd79bbdac', 'request_id': 'req-2a9a327b-eed4-48fe-a371-29f2f14cb83c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:20.234102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2da94a14ca94e13b067ae9fd79bbdac', 'request_id': 'req-2a9a327b-eed4-48fe-a371-29f2f14cb83c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:20.252408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] The token cache is being invalidate because project f2da94a14ca94e13b067ae9fd79bbdac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:20.252710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:20.253515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-2a9a327b-eed4-48fe-a371-29f2f14cb83c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:20.258981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1132/2257] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:20 2022] DELETE /identity/v3/projects/f2da94a14ca94e13b067ae9fd79bbdac => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:20.260156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.260440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.260716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.288214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:20.289452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.363624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1126/2258] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:20.371017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.371315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.371652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.393567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:20.395099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.410274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1133/2259] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:20.412527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.423267 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee40a71-5a43-4f17-bb8a-40997224875c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.428057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.428795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.428995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.429277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.430345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:20.430345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.431937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.451648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1127/2260] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:20.454412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.464099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0b8a4-19e2-4455-8c2a-0030d969290b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.469581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.470251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.470534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.470798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.471787 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.474093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.508924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1134/2261] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:58:20.510126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.521417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62074-bbd7-40d8-9715-197000f275a4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.588404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.589181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.589465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.589731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.590541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.592847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.632304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1128/2262] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:20.634064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.665289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99bf9e-4ca7-44ac-8b98-1e9233fb4b0e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.670806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.672008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.672480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.672947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.674243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.677268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.691830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1135/2263] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:20.696412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.708329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177822dc-1867-44c5-876c-214e48918159 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.713443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.714198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.714552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.714837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.717067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.722042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01144b12-f9d5-468d-8161-1bb80017aceb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.735393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1129/2264] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:20.740483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.751847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183569c-86f4-4d53-8e36-ac39de0cac20 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.756793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.757703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.758060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.758339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] PATH_INFO: `/v3/projects/fef76eedbb014e64a2a61df170001185/users/f56ddee9ed6b403f947205b533a89a5a/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.775465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef76eedbb014e64a2a61df170001185, user_id=f56ddee9ed6b403f947205b533a89a5a, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:20.778400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:20.793299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1136/2265] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:20 2022] PUT /identity/v3/projects/fef76eedbb014e64a2a61df170001185/users/f56ddee9ed6b403f947205b533a89a5a/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:20.804111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc42bd-ff71-489b-a250-a3842a427e86 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.804404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc42bd-ff71-489b-a250-a3842a427e86 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.804715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc42bd-ff71-489b-a250-a3842a427e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.837232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-59dc42bd-ff71-489b-a250-a3842a427e86 None None] MFA Rules not processed for user `f56ddee9ed6b403f947205b533a89a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:20.838533 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-59dc42bd-ff71-489b-a250-a3842a427e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.882962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:20.908594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1130/2266] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:20.919261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:20.921517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:20.986648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408c9a-3b66-4b11-9a28-2e99d0cf5d6a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:20.990248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:20.990903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:20.991413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:20.991688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:20.998397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:21.001749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:21.004094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:21.015188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1137/2267] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:22.379796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4320-2c69-47b0-8d77-05e10f0eeca9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.380328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4320-2c69-47b0-8d77-05e10f0eeca9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.380438 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e4320-2c69-47b0-8d77-05e10f0eeca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.399074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d38e4320-2c69-47b0-8d77-05e10f0eeca9 None None] MFA Rules not processed for user `f56ddee9ed6b403f947205b533a89a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:22.400324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d38e4320-2c69-47b0-8d77-05e10f0eeca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.408101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1131/2268] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:22.638498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.640215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.713735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219072b0-be02-4598-b70e-5e4bd886b7a4 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:22.719291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:22.720757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.721144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.721599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] PATH_INFO: `/v3/users/9b963031ee5a45e48b38cbb912535b2e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.723145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b963031ee5a45e48b38cbb912535b2e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:22.725314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:22.776432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b963031ee5a45e48b38cbb912535b2e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:22.789885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b963031ee5a45e48b38cbb912535b2e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:22.790715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b963031ee5a45e48b38cbb912535b2e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:22.798152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b963031ee5a45e48b38cbb912535b2e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:22.806054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b963031ee5a45e48b38cbb912535b2e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:22.813784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1138/2269] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:22 2022] DELETE /identity/v3/users/9b963031ee5a45e48b38cbb912535b2e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:22.818369 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:22.818654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:22.818928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:22.836784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:22.838195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.902165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1132/2270] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:22.914000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:22.917293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:22.985398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95311ee-0b71-4a55-8f5b-25be4157545e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.003349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.004103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.004390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.004666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.015207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.017302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4678ed-1d02-4b6b-8849-097304992dd4 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.031436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1139/2271] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:58:23.369028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.370913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:23.389701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbdd88-3ec4-467c-9f21-f46b9448f221 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.396954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.398214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.398663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.399150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] PATH_INFO: `/v3/projects/b933118abf2e4260a5d1b78b5a2ae452` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.400962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b933118abf2e4260a5d1b78b5a2ae452)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.403584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.468813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b933118abf2e4260a5d1b78b5a2ae452', 'request_id': 'req-0a07e367-9386-433b-8eb1-34ef77133244'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:23.477597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b933118abf2e4260a5d1b78b5a2ae452', 'request_id': 'req-0a07e367-9386-433b-8eb1-34ef77133244'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:23.484443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b933118abf2e4260a5d1b78b5a2ae452', 'request_id': 'req-0a07e367-9386-433b-8eb1-34ef77133244'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:23.502470 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] The token cache is being invalidate because project b933118abf2e4260a5d1b78b5a2ae452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:23.502909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:23.503647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0a07e367-9386-433b-8eb1-34ef77133244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:23.505319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1133/2272] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:23 2022] DELETE /identity/v3/projects/b933118abf2e4260a5d1b78b5a2ae452 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:23.510439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.510734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.511011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.532410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:23.533867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:23.549141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:23.615155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1140/2273] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:23.623511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.623801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.624104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.669539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:23.670967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:23.685995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1134/2274] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:23.691049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.703534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573aa123-47a1-4e29-9f6d-4ec971800314 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.708276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.708947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.709230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.709509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.710249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:23.710556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.712203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.725061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1141/2275] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:58:23.729423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.739361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757816da-110e-49a3-8132-6ff7e73d5618 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.744912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.745714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.745932 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.746250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.747048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.749076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.779901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1135/2276] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:23.783580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.795389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0641-197c-4deb-aeee-83716eec3995 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.854717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.855405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.855780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.855996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.856905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.858906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.880578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1142/2277] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:23 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:23.884491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.895616 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bd2cf-4270-41f0-8539-76d813ef8f81 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.900681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.903496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.903781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.904105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.904843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.906613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.919537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1136/2278] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:23.921679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.931839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea734-f0c1-429b-95b1-bbf6390c8b78 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.937162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.937828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.938110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.938390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.939132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:23.940859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed4835-04b6-48fa-b701-744e59b4f1c2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:23.951345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1143/2279] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:23.961213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:23.974817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70602dd7-bf60-4852-815a-b2b65a788f5a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:23.980022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:23.980909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:23.981213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:23.981490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] PATH_INFO: `/v3/projects/a1f9b16833d64907a4677a189b3b8c83/users/ed15785c2b294e62ba39e41724aebeae/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:23.997668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f9b16833d64907a4677a189b3b8c83, user_id=ed15785c2b294e62ba39e41724aebeae, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.000629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.004564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:24.018641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1137/2280] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:23 2022] PUT /identity/v3/projects/a1f9b16833d64907a4677a189b3b8c83/users/ed15785c2b294e62ba39e41724aebeae/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:24.030165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.030456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.030735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.058541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] MFA Rules not processed for user `ed15785c2b294e62ba39e41724aebeae`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:24.059801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:24.133510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1144/2281] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:24.145503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:24.147638 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:24.213825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aacdec-6c56-4120-ad2d-a5d2090bae8a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:24.217985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:24.218893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:24.219276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:24.219683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:24.236386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:24.237816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a090c93-9422-4237-a7dc-43d8f2bc10d0 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:24.250993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1138/2282] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:33.563181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.580555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:33.678950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80cfa8-f973-4492-af3d-de0a85d8ec43 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.686152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.687153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.687590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.688011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] PATH_INFO: `/v3/users/ed15785c2b294e62ba39e41724aebeae` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.689552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed15785c2b294e62ba39e41724aebeae)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.691906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:33.751069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed15785c2b294e62ba39e41724aebeae'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:33.764461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed15785c2b294e62ba39e41724aebeae'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:33.766412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed15785c2b294e62ba39e41724aebeae'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:33.773346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed15785c2b294e62ba39e41724aebeae'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:33.782829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed15785c2b294e62ba39e41724aebeae'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:33.790158 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1145/2283] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:33 2022] DELETE /identity/v3/users/ed15785c2b294e62ba39e41724aebeae => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:33.794652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.795140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.795282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.812374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:33.813609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:33.904674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1139/2284] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:33.915967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:33.916481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:33.980972 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cd625-af35-4cf0-847d-ce02ee1028ea admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:33.985657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:33.986316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:33.986601 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:33.986878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:33.997337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:33.998777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3860768-0223-4e49-955a-c39a8348d39f service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.009585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1146/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:58:34.368847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.370307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:34.382301 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6908-9da4-4e37-a62b-d888928c3ead None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.386954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.387704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.388023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.388290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] PATH_INFO: `/v3/projects/a1f9b16833d64907a4677a189b3b8c83` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.389471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f9b16833d64907a4677a189b3b8c83)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.391775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.446645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f9b16833d64907a4677a189b3b8c83', 'request_id': 'req-4ebceacd-47db-4154-b299-25ccfcf2f1d3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:34.452577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f9b16833d64907a4677a189b3b8c83', 'request_id': 'req-4ebceacd-47db-4154-b299-25ccfcf2f1d3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:34.457949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f9b16833d64907a4677a189b3b8c83', 'request_id': 'req-4ebceacd-47db-4154-b299-25ccfcf2f1d3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:34.474065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] The token cache is being invalidate because project a1f9b16833d64907a4677a189b3b8c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:34.474393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:34.475104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4ebceacd-47db-4154-b299-25ccfcf2f1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:34.476953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1140/2286] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:34 2022] DELETE /identity/v3/projects/a1f9b16833d64907a4677a189b3b8c83 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:34.482807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.483051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.483374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.501988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:34.503193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:34.533930 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.570706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1147/2287] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:34.578874 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.579160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.579437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.599153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:34.600322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:34.610632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1141/2288] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:34.614899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.626249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c2c8d-e879-4339-b387-db679097a7eb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.632123 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.632806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.633119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.633394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.634269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:58:34.635161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.636830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.654295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1148/2289] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:58:34.656108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.666751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613e3b7-f523-49d5-bed3-085514b5af10 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.671613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.672466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.673064 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.673391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.674178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.676245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.691566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:34.716180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1142/2290] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:34.717955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.729373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9125f1b-ea63-4f33-ae61-a1a518c6bb4c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.800837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.801503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.801771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.802042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.802732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.804677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.828903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1149/2291] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:34.830123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.841089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c9ad-9314-4e7a-aa2d-cd84f643fb37 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.846483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.847152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.847433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.851263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.852149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.854073 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.870193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1143/2292] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:34.871423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.882421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514493-3354-4b6e-9911-bd6a00c71eb2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.887790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.888610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.889164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.889463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.890366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.892110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033777a-205e-4a4e-baaf-2e013a9b4389 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.903924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1150/2293] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:34.908222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:34.921393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffe489-64fb-4601-9bc5-743635b1584a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:34.927347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:34.928582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.929070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.929546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] PATH_INFO: `/v3/projects/3d2953bf079541b28189cc9ad88ad753/users/600fb1135fe040e2aa1c03628da8bf93/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.949696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2953bf079541b28189cc9ad88ad753, user_id=600fb1135fe040e2aa1c03628da8bf93, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:34.952909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:34.972697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1144/2294] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:34 2022] PUT /identity/v3/projects/3d2953bf079541b28189cc9ad88ad753/users/600fb1135fe040e2aa1c03628da8bf93/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:34.977886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0440-7316-4e0f-b2ab-e83a346fb9a7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:34.978179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0440-7316-4e0f-b2ab-e83a346fb9a7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:34.978461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0440-7316-4e0f-b2ab-e83a346fb9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:34.996182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-acab0440-7316-4e0f-b2ab-e83a346fb9a7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:34.997514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-acab0440-7316-4e0f-b2ab-e83a346fb9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:35.080830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1151/2295] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:35.091404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:35.093331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:35.159056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab425df-d349-4cad-a42a-92d16fb6bc25 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:35.162820 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:35.163588 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:35.163911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:35.164177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:35.175381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:35.176902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085f1f-2c62-49a4-9ecf-e7c15f876fcb service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:35.188255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1145/2296] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:39.743685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.744210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.744302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.768221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] MFA Rules not processed for user `a7d7f7af47db49d9a30db75c76980cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:39.769756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:39.829476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:39.856614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1152/2297] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:39.953827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:39.954242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:39.954393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:39.994287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] MFA Rules not processed for user `600fb1135fe040e2aa1c03628da8bf93`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:39.995684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:40.091296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1146/2298] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:40.096101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.113999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308725-9b5f-472e-88e6-fe0c97624c69 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.119205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.119937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.120224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.120505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.121308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.123405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.152643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1153/2299] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:58:40 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:58:40.156913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.168941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a6c7-f8df-41a5-9073-6388fdf0be8e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.234742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:40.255708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.256495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.256779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.257056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.257771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.259772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.291919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1147/2300] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:58:40 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:58:40.294580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.306082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341841d-326d-4d16-8001-a4fe245c8352 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.311347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.312184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.312476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.312841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.313600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.315235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.333209 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.340230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1154/2301] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:40.344172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693f2f3-db06-4cc4-a4de-081638946ea8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.350352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.351341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.351693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.352072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] PATH_INFO: `/v3/projects/4100a1d9888c420c9e36353196577e3b/users/388af5ed115a40038c0674b1cba10dc2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.373522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4100a1d9888c420c9e36353196577e3b, user_id=388af5ed115a40038c0674b1cba10dc2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.376787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.392283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1148/2302] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:40 2022] PUT /identity/v3/projects/4100a1d9888c420c9e36353196577e3b/users/388af5ed115a40038c0674b1cba10dc2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:40.397131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.408485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6539d31-4234-4a11-8dba-8fe7aa233c2b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.478239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.479111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.479610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.497988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.498660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.500556 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab941d0-1ccf-4f01-b38b-e5e4287a3e73 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.528686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1155/2303] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:58:40.529957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:40.544063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cba9302-ab70-4bd6-8278-27894c5ef84f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:40.550977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:40.551909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:40.552198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:40.552468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] PATH_INFO: `/v3/projects/4100a1d9888c420c9e36353196577e3b/users/388af5ed115a40038c0674b1cba10dc2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:40.560448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4100a1d9888c420c9e36353196577e3b, user_id=388af5ed115a40038c0674b1cba10dc2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:40.563296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:40.579033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1149/2304] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:58:40 2022] PUT /identity/v3/projects/4100a1d9888c420c9e36353196577e3b/users/388af5ed115a40038c0674b1cba10dc2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:45.163484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67eccf-64e9-4ca1-8e0c-be0cbeb89393 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.164018 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67eccf-64e9-4ca1-8e0c-be0cbeb89393 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.164117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67eccf-64e9-4ca1-8e0c-be0cbeb89393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.183631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0c67eccf-64e9-4ca1-8e0c-be0cbeb89393 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:45.184974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0c67eccf-64e9-4ca1-8e0c-be0cbeb89393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:45.259219 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1156/2305] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:58:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:45.330667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:45.339473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:45.447486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a3534-3c3f-40a3-bd68-f79f1fb20d6d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:45.456174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:45.457678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:45.457678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:45.457678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:45.470349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:45.477323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:45.490989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1150/2306] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:58:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:58:46.170222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07a53-88be-4c17-b267-bc140dd6e74c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.170745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07a53-88be-4c17-b267-bc140dd6e74c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.170946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a07a53-88be-4c17-b267-bc140dd6e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.190106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-66a07a53-88be-4c17-b267-bc140dd6e74c None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:46.191415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-66a07a53-88be-4c17-b267-bc140dd6e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:46.199151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1157/2307] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 10:58:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:46.212435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.222949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e142f2-286b-43cd-ba79-bdbd14961769 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.228150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.228809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.229091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.229356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.239005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.240664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.249378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1151/2308] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:58:46.512879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e439-fa40-45f4-866b-715262457fe5 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.513366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e439-fa40-45f4-866b-715262457fe5 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.513644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e439-fa40-45f4-866b-715262457fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.558947 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0c06e439-fa40-45f4-866b-715262457fe5 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:46.561121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0c06e439-fa40-45f4-866b-715262457fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:46.580980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1158/2309] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:58:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:46.599293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:46.613287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1ff69-cf76-497b-a0be-8dc936718d79 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:46.622601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:46.623534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:46.623992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:46.624409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:46.641714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:46.645221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:46.683665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1152/2310] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:58:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:58:47.232512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8accb2-c940-4d99-acff-5300bb332e87 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.233348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8accb2-c940-4d99-acff-5300bb332e87 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.233789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8accb2-c940-4d99-acff-5300bb332e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.259829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-da8accb2-c940-4d99-acff-5300bb332e87 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:47.261597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-da8accb2-c940-4d99-acff-5300bb332e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:47.271003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1159/2311] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 10:58:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:47.293037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:47.314147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dfc1-689e-45c0-ae09-2b85b1d67487 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:47.329115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:47.331149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:47.331536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:47.332081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:47.346478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:47.349900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:47.387951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1153/2312] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:58:49.296884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.297536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.298091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.332088 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] MFA Rules not processed for user `388af5ed115a40038c0674b1cba10dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:49.333375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:49.375582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.456109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1160/2313] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:49.774498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:49.776899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:49.829435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:49.852510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c17fb-c234-4bda-b21f-b26d0c91040a service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:49.858342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:49.859052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:49.859335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:49.859611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:49.923240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:49.925625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf134-7e2a-4bab-9243-9ea6c6a41cd1 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:49.935782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1154/2314] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:50.307098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.308714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:50.384106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208b850-5a50-467a-9c7e-45bfaaa52715 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.388668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.389523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.389807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.390081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] PATH_INFO: `/v3/users/f56ddee9ed6b403f947205b533a89a5a` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.391951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56ddee9ed6b403f947205b533a89a5a)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.393896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.449903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56ddee9ed6b403f947205b533a89a5a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:50.464750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56ddee9ed6b403f947205b533a89a5a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:50.465512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56ddee9ed6b403f947205b533a89a5a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:50.473866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56ddee9ed6b403f947205b533a89a5a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:50.482868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56ddee9ed6b403f947205b533a89a5a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:50.494377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1161/2315] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:58:50 2022] DELETE /identity/v3/users/f56ddee9ed6b403f947205b533a89a5a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:58:50.496167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc326-057d-495c-baf0-3f9f1f848d7e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.496555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc326-057d-495c-baf0-3f9f1f848d7e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.496757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddc326-057d-495c-baf0-3f9f1f848d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.518111 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-18ddc326-057d-495c-baf0-3f9f1f848d7e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:50.519747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-18ddc326-057d-495c-baf0-3f9f1f848d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:50.607954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1155/2316] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:50.636016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:50.638170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:50.699805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:50.721114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e229203-3b69-488d-b2d8-1d62c4662853 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:50.726263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:50.727271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:50.727625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:50.728232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:50.743064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:50.744777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:50.758679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1162/2317] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:58:51.001902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23e4c-b1ad-4429-9862-ed77c26740be None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.003739 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23e4c-b1ad-4429-9862-ed77c26740be None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.004422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de23e4c-b1ad-4429-9862-ed77c26740be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.045746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7de23e4c-b1ad-4429-9862-ed77c26740be None None] MFA Rules not processed for user `b47991b6880c4a91ab42d2567eee61aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:51.047144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7de23e4c-b1ad-4429-9862-ed77c26740be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.106808 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:51.108425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.129587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47bb43-06c2-4b38-960d-9440b2e4e105 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:51.138342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:51.139305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.139663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.140137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] PATH_INFO: `/v3/projects/fef76eedbb014e64a2a61df170001185` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.142705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef76eedbb014e64a2a61df170001185)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:51.159650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:51.179535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1156/2318] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:51.232820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef76eedbb014e64a2a61df170001185', 'request_id': 'req-fc22a1f6-9577-40d2-877b-5c61fe4c3305'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:51.244237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef76eedbb014e64a2a61df170001185', 'request_id': 'req-fc22a1f6-9577-40d2-877b-5c61fe4c3305'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:51.254295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef76eedbb014e64a2a61df170001185', 'request_id': 'req-fc22a1f6-9577-40d2-877b-5c61fe4c3305'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:51.272636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] The token cache is being invalidate because project fef76eedbb014e64a2a61df170001185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:58:51.273028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:51.273776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fc22a1f6-9577-40d2-877b-5c61fe4c3305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:58:51.275606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1163/2319] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:58:51 2022] DELETE /identity/v3/projects/fef76eedbb014e64a2a61df170001185 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:58:51.590644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:51.591849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:51.591849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:51.609558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] MFA Rules not processed for user `600fb1135fe040e2aa1c03628da8bf93`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:51.611282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:51.650507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:58:51.700854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1157/2320] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:55.115580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b99d-2061-4770-a0b3-21f8b20079d7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.116122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b99d-2061-4770-a0b3-21f8b20079d7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.116269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b99d-2061-4770-a0b3-21f8b20079d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.136437 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4e87b99d-2061-4770-a0b3-21f8b20079d7 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:58:55.137658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4e87b99d-2061-4770-a0b3-21f8b20079d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:55.199207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1164/2321] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:58:55.212920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:58:55.214334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:58:55.291172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9275921-14ce-4976-92d1-a2f6392730c9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:58:55.298979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:58:55.299993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:58:55.300433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:58:55.300860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:58:55.311476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:58:55.313270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:58:55.321984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1158/2322] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:58:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:59:05.192198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:05.319040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=41 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:11.161256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804abbe1-b682-4251-bf6c-280b2be8d6c9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.161918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804abbe1-b682-4251-bf6c-280b2be8d6c9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.161918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804abbe1-b682-4251-bf6c-280b2be8d6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.184150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-804abbe1-b682-4251-bf6c-280b2be8d6c9 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:11.185550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-804abbe1-b682-4251-bf6c-280b2be8d6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:11.194738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1165/2323] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:59:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:11.210905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:11.223056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359544f-4e19-4d48-afe6-b8cd47f97971 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:11.230507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:11.231217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:11.231655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:11.231994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:11.243415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:11.245088 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55e1f5-d1a1-44c8-a549-83f947273891 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:11.254908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1159/2324] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:59:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:59:13.846650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.847645 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.848100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.850623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1166/2325] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:59:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:13.857901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2655e-5bc7-4f2c-b9be-6048510716e7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:13.858728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2655e-5bc7-4f2c-b9be-6048510716e7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:13.859191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2655e-5bc7-4f2c-b9be-6048510716e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:13.892381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ddd2655e-5bc7-4f2c-b9be-6048510716e7 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:13.894328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ddd2655e-5bc7-4f2c-b9be-6048510716e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.057520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1160/2326] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:14.070557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.072499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.186056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330edf7-ed8b-48a6-8cc2-644991f66e4b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.208434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.209173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.209460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.209734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.232991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.234566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd7787-2861-491b-bf0a-dcc243046dcf service glance] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.264370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1167/2327] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:14.414962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.415606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.416570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.418930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1161/2328] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:59:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:14.424336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b2f2b-6c92-43a2-9317-4aebb4c7078f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.424737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b2f2b-6c92-43a2-9317-4aebb4c7078f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.425232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b2f2b-6c92-43a2-9317-4aebb4c7078f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.457665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-d91b2f2b-6c92-43a2-9317-4aebb4c7078f None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:14.459544 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d91b2f2b-6c92-43a2-9317-4aebb4c7078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.542408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1168/2329] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:14.563764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.566942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.586280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94facd4-41d9-4fbe-9c96-e2c030ab5fb4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.590364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.591029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.591313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.591751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.604344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.608371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95846cb-defa-4cc4-8c11-45eea3dd5879 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.628215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1162/2330] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:14.833131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f5a8-5c2c-4abd-b39b-7e773c77228f None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.833613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f5a8-5c2c-4abd-b39b-7e773c77228f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.833846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f5a8-5c2c-4abd-b39b-7e773c77228f None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.836163 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1169/2331] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 10:59:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:59:14.853866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086f05-fa1a-4d1e-be42-e514df9a2060 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.854230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086f05-fa1a-4d1e-be42-e514df9a2060 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.854539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086f05-fa1a-4d1e-be42-e514df9a2060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.885381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-89086f05-fa1a-4d1e-be42-e514df9a2060 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:14.887985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-89086f05-fa1a-4d1e-be42-e514df9a2060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:14.925413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1163/2332] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:14.939500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f5a8-5c2c-4abd-b39b-7e773c77228f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:14.957595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f5a8-5c2c-4abd-b39b-7e773c77228f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:14.963028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:14.963924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:14.964360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:14.964820 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:14.978534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:14.980350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:14.993317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1170/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:18.862103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26107a0-4f39-4938-8a64-bf4a7e74436d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.862701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26107a0-4f39-4938-8a64-bf4a7e74436d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.862808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26107a0-4f39-4938-8a64-bf4a7e74436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.884362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a26107a0-4f39-4938-8a64-bf4a7e74436d None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:18.885883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a26107a0-4f39-4938-8a64-bf4a7e74436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:18.903641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1164/2334] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:59:18.924243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:18.935868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191cac7-8430-4ddf-ac0f-d8608f64ba86 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:18.948549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:18.949563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:18.949852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:18.950217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:18.965086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:18.967220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:18.978223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1171/2335] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:59:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:59:28.878399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=49 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:28.935796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:28.936135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:28.936421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:28.961528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:28.964729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:28.972706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1165/2336] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 10:59:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:28.975144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=64 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:28.987247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:28.999984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0b967-67d3-4ccf-9c38-1c3922d84f68 service swift] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:29.006021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:29.006866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:29.007062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:29.007580 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:29.018286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:29.020045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:29.029712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1172/2337] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:59:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:59:31.106978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.109091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:31.181265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538766b1-7327-496f-a71a-725dfed7ebe9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.185845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.186707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.186972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.187242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] PATH_INFO: `/v3/users/a7d7f7af47db49d9a30db75c76980cb9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.188553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d7f7af47db49d9a30db75c76980cb9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.190462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.240561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d7f7af47db49d9a30db75c76980cb9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.254239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d7f7af47db49d9a30db75c76980cb9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.254966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d7f7af47db49d9a30db75c76980cb9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.263744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d7f7af47db49d9a30db75c76980cb9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.272577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-95849608-103a-4e76-88f3-4498b63c1ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d7f7af47db49d9a30db75c76980cb9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.278564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1166/2338] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 10:59:31 2022] DELETE /identity/v3/users/a7d7f7af47db49d9a30db75c76980cb9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:31.529166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.540048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6cd75-3316-4d50-b290-75c0b8752f99 service swift] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.608797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.609669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.609959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.610233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] PATH_INFO: `/v3/projects/8a913265f84c43579ce0fb5a2604b663` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.611907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a913265f84c43579ce0fb5a2604b663)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.613837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.669637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a913265f84c43579ce0fb5a2604b663', 'request_id': 'req-5a346380-259b-4cb5-be87-97cd50a56bf2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.675517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a913265f84c43579ce0fb5a2604b663', 'request_id': 'req-5a346380-259b-4cb5-be87-97cd50a56bf2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.682465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a913265f84c43579ce0fb5a2604b663', 'request_id': 'req-5a346380-259b-4cb5-be87-97cd50a56bf2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.695997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] The token cache is being invalidate because project 8a913265f84c43579ce0fb5a2604b663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:59:31.696281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.696882 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5a346380-259b-4cb5-be87-97cd50a56bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:59:31.698417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1173/2339] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 10:59:31 2022] DELETE /identity/v3/projects/8a913265f84c43579ce0fb5a2604b663 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:31.704754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.705036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.705307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.723118 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:31.724332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:31.791109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1167/2340] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:31.799150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.799477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.799752 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.820054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:31.821257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:31.830105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1174/2341] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:31.834834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.848326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995d144-20da-460e-9005-8e18b20e3acf None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.853886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.854548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.854832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.855114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.855846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:59:31.856191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.857817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.866819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:31.870473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1168/2342] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 10:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:59:31.875238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.885220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9b5f-5280-45ae-b554-39615c7dbdcb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.890377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.891062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.891366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.891714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:31.892406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:31.894306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:31.924506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1175/2343] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 10:59:31 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:31.928871 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:31.939386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b51af0-6e68-4ea6-9a51-e7e61162d8e5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:31.998653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:31.999318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:31.999605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:31.999891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.000608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.002711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.025407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1169/2344] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 10:59:31 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:59:32.029727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.040825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606f59-0417-4b91-b764-350efdb0c051 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.045951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.046546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.046831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.047122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.047911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.049666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.060398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1176/2345] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:59:32.064747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.074674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef92e6-6059-4da5-986d-78ef514a2b10 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.079339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.080060 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.080355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.080643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.081357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.082966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d19aba-80e4-4c59-be83-6cc74ab64e92 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.092756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1170/2346] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 10:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:59:32.097004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.107188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ba666-a3dd-4f89-a2f8-8859c717ed94 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.114298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.115605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.116058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.116411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] PATH_INFO: `/v3/projects/093d935c7e104ba0a6cb02c87a67a173/users/5e2403128f074fed9e96d556cdeaf0c3/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.134415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=093d935c7e104ba0a6cb02c87a67a173, user_id=5e2403128f074fed9e96d556cdeaf0c3, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.137102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.153039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1177/2347] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 10:59:32 2022] PUT /identity/v3/projects/093d935c7e104ba0a6cb02c87a67a173/users/5e2403128f074fed9e96d556cdeaf0c3/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:59:32.163023 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.163541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.163646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.189895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] MFA Rules not processed for user `5e2403128f074fed9e96d556cdeaf0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:32.191084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:32.259156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1171/2348] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 10:59:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:32.270968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:32.273836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:32.289526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:32.337652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654f62a-0bb3-47fd-b262-cef9c3f6ba7f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:32.341125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:32.341771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:32.342051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:32.342325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:32.351073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:32.352535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54712c39-b517-4286-a8f4-f13ccc6b5ee6 service glance] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:32.361764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1178/2349] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:40.129161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:40.130675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:40.200052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f8df6-e992-4203-8dd2-1447cd4b6df4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:40.203634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:40.204386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:40.204669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:40.204943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:40.206863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:40.229163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:40.281534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:40.283317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:40.293107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1172/2350] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 10:59:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:59:42.357523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=20 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:44.875797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:44.876970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:44.876970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:44.894940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:59:44.896253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:44.957587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1179/2351] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 10:59:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:59:44.970614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:59:44.972295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:59:45.042930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fb5fb-71b5-420c-886e-2b13ff7fea24 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:59:45.049336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:59:45.050042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:59:45.050348 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:59:45.050620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:59:45.060373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:59:45.061995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5b05-82e2-4687-bce0-c53990c61fb7 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:59:45.070545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1173/2352] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 10:59:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:59:54.949739 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=25 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:59:55.067216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=51 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:04.804619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:04.870478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdae67-b74e-44ba-88eb-a951fed15271 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:04.876082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:04.877485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.877775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.878043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] PATH_INFO: `/v3/users/5e2403128f074fed9e96d556cdeaf0c3` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.879265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2403128f074fed9e96d556cdeaf0c3)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:04.881249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:04.931577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2403128f074fed9e96d556cdeaf0c3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:04.944336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2403128f074fed9e96d556cdeaf0c3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:04.945067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2403128f074fed9e96d556cdeaf0c3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:04.951693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2403128f074fed9e96d556cdeaf0c3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:04.960853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2403128f074fed9e96d556cdeaf0c3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:04.966896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1180/2353] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:00:04 2022] DELETE /identity/v3/users/5e2403128f074fed9e96d556cdeaf0c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:04.971537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:04.972049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:04.972165 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:04.989988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:04.991208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:05.059651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1174/2354] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:05.069966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.071499 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:05.134932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b5449-283a-4384-85bc-03bb16a144ea admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.138976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.139665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.139993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.140274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.151116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.152900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80a6fe-31f8-40c4-a4f6-4470ea72b55c service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.163391 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1181/2355] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:05.457252 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.458959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:05.472229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637071-e185-4147-98ad-ebba51b17f37 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.479572 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.480464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.480705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.480992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] PATH_INFO: `/v3/projects/093d935c7e104ba0a6cb02c87a67a173` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.482498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093d935c7e104ba0a6cb02c87a67a173)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.484610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.534991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093d935c7e104ba0a6cb02c87a67a173', 'request_id': 'req-6d69b587-d99b-4c6d-9646-ceed5bc451d1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:05.541427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093d935c7e104ba0a6cb02c87a67a173', 'request_id': 'req-6d69b587-d99b-4c6d-9646-ceed5bc451d1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:05.547476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093d935c7e104ba0a6cb02c87a67a173', 'request_id': 'req-6d69b587-d99b-4c6d-9646-ceed5bc451d1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:05.564312 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] The token cache is being invalidate because project 093d935c7e104ba0a6cb02c87a67a173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:00:05.564645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:05.565842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6d69b587-d99b-4c6d-9646-ceed5bc451d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:05.568381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1175/2356] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:00:05 2022] DELETE /identity/v3/projects/093d935c7e104ba0a6cb02c87a67a173 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:05.573686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.574089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.574425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.593837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:05.595112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:05.619417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:05.669065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1182/2357] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:05.678168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.678449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.678717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.698631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:05.699824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:05.709000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1176/2358] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:05.714918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.725282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856f80d-5ca9-4ff1-b5be-a9cb207a285e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.730338 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.731002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.731277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.731549 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.732309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:05.732613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.734217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.753834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1183/2359] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:05.758511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.770554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4da4f-e5ee-438e-8b4e-9387c182d733 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.779175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.779935 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.780313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.780652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.781648 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.785276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.814019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1177/2360] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:00:05 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:00:05.818577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.829411 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9549-548d-4e3b-91e7-ed65a28606f1 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.890631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.891327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.891612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.891897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.892612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.894597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.918584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1184/2361] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:00:05 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:05.922629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.933576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce02c9-4f1f-4adf-845e-df4346f285a8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.939275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.939997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.940279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.940548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.941297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.942923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.954194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1178/2362] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:00:05.958662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:05.969151 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f9c7-18db-4732-90af-e2e9b662849f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:05.974452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:05.975112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:05.975403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:05.975682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:05.976456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:05.978104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb9080-ce1c-4ace-a16e-9471ecd4e821 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:05.989234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1185/2363] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:00:05.993439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:06.004196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4185d0-af11-45f2-9d2a-7c0e7de64ee5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:06.009192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:06.010069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.010350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.010607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] PATH_INFO: `/v3/projects/8f916b47907c4c2ba42e7c1a70c58971/users/51be67277da14acbbee58dbb135c51ba/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.026735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f916b47907c4c2ba42e7c1a70c58971, user_id=51be67277da14acbbee58dbb135c51ba, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:06.029417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:06.044611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1179/2364] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:00:05 2022] PUT /identity/v3/projects/8f916b47907c4c2ba42e7c1a70c58971/users/51be67277da14acbbee58dbb135c51ba/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:06.057811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.058154 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.058519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.090531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] MFA Rules not processed for user `51be67277da14acbbee58dbb135c51ba`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:06.091654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:06.165306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1186/2365] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:06.177421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:06.179504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:06.201742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:06.245487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419efb70-d677-44e9-9c25-d3a327566b1a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:06.249356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:06.250175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:06.250463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:06.250782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:06.260545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:06.262030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7489d2-52bd-40b2-b1b3-a9887ed6061a service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:06.271553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1180/2366] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:08.815865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:08.836199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:08.882465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fabbd-cb95-45db-ba9a-76ade61b1494 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:08.887278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:08.888195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:08.888498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:08.888767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] PATH_INFO: `/v3/users/51be67277da14acbbee58dbb135c51ba` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:08.890065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51be67277da14acbbee58dbb135c51ba)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:08.892045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:08.957695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51be67277da14acbbee58dbb135c51ba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:08.970374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51be67277da14acbbee58dbb135c51ba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:08.971035 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51be67277da14acbbee58dbb135c51ba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:08.978353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51be67277da14acbbee58dbb135c51ba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:08.987484 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51be67277da14acbbee58dbb135c51ba'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:08.993867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1187/2367] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:00:08 2022] DELETE /identity/v3/users/51be67277da14acbbee58dbb135c51ba => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:08.999950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.000439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.000657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.018643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:09.019923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.083907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1181/2368] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:09.093952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.095594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.160061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225957a7-05d1-41ba-bade-02b504f06b15 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.164693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.165344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.165623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.165904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.175443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.177465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d6bc-0f75-4e5e-b50c-8208f0f914f7 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.187055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1188/2369] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:09.485926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.487416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.500967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d056e48-ab80-4dc9-870c-74e1db22771c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.507260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.508184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.508477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.508752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] PATH_INFO: `/v3/projects/8f916b47907c4c2ba42e7c1a70c58971` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.510103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f916b47907c4c2ba42e7c1a70c58971)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.512190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.562975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f916b47907c4c2ba42e7c1a70c58971', 'request_id': 'req-0692ae9c-1a73-409f-bdfd-57070ffab135'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:09.569755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f916b47907c4c2ba42e7c1a70c58971', 'request_id': 'req-0692ae9c-1a73-409f-bdfd-57070ffab135'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:09.576846 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f916b47907c4c2ba42e7c1a70c58971', 'request_id': 'req-0692ae9c-1a73-409f-bdfd-57070ffab135'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:09.592490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] The token cache is being invalidate because project 8f916b47907c4c2ba42e7c1a70c58971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:00:09.592780 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:09.593423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0692ae9c-1a73-409f-bdfd-57070ffab135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:09.594991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1182/2370] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:00:09 2022] DELETE /identity/v3/projects/8f916b47907c4c2ba42e7c1a70c58971 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:00:09.601329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.601620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.601900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.620835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:09.622028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.665199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:09.691719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1189/2371] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:09.700078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.700362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.700637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.719865 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:09.721069 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:09.729787 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1183/2372] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:09.734421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.744693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cadec-bf67-4ef5-8a1b-f3cf1c94d335 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.750293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.751008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.751205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.751482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.752378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:00:09.752686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.754339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.766634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1190/2373] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:00:09.771316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.781208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc85a78-258c-45d3-9490-42cdc7f92ab4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.786423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.787085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.787370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.787679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.788418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.790352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.819582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1184/2374] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:00:09 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:09.825530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.838318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aac986-6fce-462e-8dce-97e2f916345f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.896467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.897149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.897434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.897694 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.898715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.900725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.923007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1191/2375] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:00:09 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:00:09.927481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.938203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d551b-4a84-4c98-9b93-a1882ef9055f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.943329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.944055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.944344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.944624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.945475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.947216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.959975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1185/2376] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:00:09.963863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:09.974327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae5ab1-d268-4fbd-9ec2-d899c5eea45a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:09.979213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:09.979910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:09.980204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:09.980879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:09.981406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:09.983031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563611ff-f6d8-41cf-a8cf-1aba0dda306a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:09.997082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1192/2377] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:00:09.998622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.016302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2156a5-f4ae-4f4b-a367-922478f46397 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.021402 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.022297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.022587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.022853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] PATH_INFO: `/v3/projects/1eda1b7054bd4878b6d433a1b6a5d219/users/905861cd1e614ec699848b2bc50ca69b/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.025622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:10.042921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eda1b7054bd4878b6d433a1b6a5d219, user_id=905861cd1e614ec699848b2bc50ca69b, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.046892 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.063813 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1186/2378] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:00:09 2022] PUT /identity/v3/projects/1eda1b7054bd4878b6d433a1b6a5d219/users/905861cd1e614ec699848b2bc50ca69b/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:10.131927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115840c-0241-4458-a811-4d8d2b6485a4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.132211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115840c-0241-4458-a811-4d8d2b6485a4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.132516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115840c-0241-4458-a811-4d8d2b6485a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.162435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f115840c-0241-4458-a811-4d8d2b6485a4 None None] MFA Rules not processed for user `905861cd1e614ec699848b2bc50ca69b`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:10.163630 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f115840c-0241-4458-a811-4d8d2b6485a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:10.230561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1193/2379] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:10.242277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:10.244370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:10.308878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e63cf-e491-4c70-b985-c372f9452c0c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:10.313114 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:10.313763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:10.314042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:10.314322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:10.323715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:10.325132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:10.334856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1187/2380] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:20.227665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:20.330123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:25.169252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bf9c-221e-4d6c-98b1-2d06cd9f9362 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:25.169735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bf9c-221e-4d6c-98b1-2d06cd9f9362 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:25.169827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bf9c-221e-4d6c-98b1-2d06cd9f9362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:25.189313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4958bf9c-221e-4d6c-98b1-2d06cd9f9362 None None] MFA Rules not processed for user `b47991b6880c4a91ab42d2567eee61aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:25.190522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4958bf9c-221e-4d6c-98b1-2d06cd9f9362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:25.250099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1194/2381] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:26.633058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.708395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466e043-7527-442a-88e0-be9329c12e8a service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:26.713469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:26.714342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.714630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.714906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] PATH_INFO: `/v3/users/905861cd1e614ec699848b2bc50ca69b` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.716261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] RBAC: Authorizing `identity:delete_user(user_id=905861cd1e614ec699848b2bc50ca69b)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:26.718203 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:26.773270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905861cd1e614ec699848b2bc50ca69b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:26.785164 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905861cd1e614ec699848b2bc50ca69b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:26.785867 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905861cd1e614ec699848b2bc50ca69b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:26.792698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905861cd1e614ec699848b2bc50ca69b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:26.802418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905861cd1e614ec699848b2bc50ca69b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:26.808433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1188/2382] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:00:26 2022] DELETE /identity/v3/users/905861cd1e614ec699848b2bc50ca69b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:26.813235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.813766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.813766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.831400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:26.832650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.901976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1195/2383] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:00:26.912019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:26.913651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:26.976920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bb079-ba42-4062-9e74-ecbff7d5051b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:26.981002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:26.981653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:26.981937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:26.982213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:26.992074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:26.993990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:27.007511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1189/2384] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:27.342071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:27.343647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:27.357424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1d47-d19f-4cc8-9c18-cfeaba321b62 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:27.362229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:27.363349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:27.363699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:27.363978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] PATH_INFO: `/v3/projects/1eda1b7054bd4878b6d433a1b6a5d219` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:27.365184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eda1b7054bd4878b6d433a1b6a5d219)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:27.367108 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:27.415124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eda1b7054bd4878b6d433a1b6a5d219', 'request_id': 'req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:27.420649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eda1b7054bd4878b6d433a1b6a5d219', 'request_id': 'req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:27.427356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eda1b7054bd4878b6d433a1b6a5d219', 'request_id': 'req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:27.440789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] The token cache is being invalidate because project 1eda1b7054bd4878b6d433a1b6a5d219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:00:27.441074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:27.441690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a31129d4-948f-4eb9-b2dd-74da98e9c2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:00:27.443227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1196/2385] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:00:27 2022] DELETE /identity/v3/projects/1eda1b7054bd4878b6d433a1b6a5d219 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:00:35.933089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:00:35.935189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:35.965259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:36.015759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cb992-fde5-4b55-aae8-d7eeddafc352 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:00:36.077510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:00:36.078154 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:36.078430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:36.078698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:36.080542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:36.091702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:00:36.093322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c29715-fdf8-40e1-ac7c-ae08a4299888 service placement] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:00:36.103214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1190/2386] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:00:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:00:37.438193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:45.009686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:00:45.010324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:00:45.010324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:00:45.029424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] MFA Rules not processed for user `3e4012911dad42edbffead00e0fd55e6`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:00:45.030659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:00:45.085895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1197/2387] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:00:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:00:46.099024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=48 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:00:55.078797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=25 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:05.649100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:05.649654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:05.649654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:05.667599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] MFA Rules not processed for user `6ead76e4f3564d6eb614bb911ccaf674`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:05.668823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:05.725654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1191/2388] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:15.718798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=25 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:27.408857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:27.410399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:27.479424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972cd13-6880-4bd4-bd5f-27e7ff0434f6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:27.484413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:27.485352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:27.485627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:27.485892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] PATH_INFO: `/v3/users/b47991b6880c4a91ab42d2567eee61aa` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:27.487271 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b47991b6880c4a91ab42d2567eee61aa)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:27.489248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:27.541199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b47991b6880c4a91ab42d2567eee61aa'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.555588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b47991b6880c4a91ab42d2567eee61aa'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.556304 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b47991b6880c4a91ab42d2567eee61aa'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.563421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b47991b6880c4a91ab42d2567eee61aa'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.573134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47403678-dd91-4a2e-961d-8e6ffa28bd6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b47991b6880c4a91ab42d2567eee61aa'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.579171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1198/2389] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:01:27 2022] DELETE /identity/v3/users/b47991b6880c4a91ab42d2567eee61aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:27.761761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:27.780995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568775b-e4f8-4590-a300-dbd4bde22040 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:27.843398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:27.844299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:27.844575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:27.844851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] PATH_INFO: `/v3/projects/c2d5301114244be2a74210484c6536ef` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:27.846076 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d5301114244be2a74210484c6536ef)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:27.848052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:27.894266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d5301114244be2a74210484c6536ef', 'request_id': 'req-eb68a9a4-a856-49a3-a5db-fed5ad314b64'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.900395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d5301114244be2a74210484c6536ef', 'request_id': 'req-eb68a9a4-a856-49a3-a5db-fed5ad314b64'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.906381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d5301114244be2a74210484c6536ef', 'request_id': 'req-eb68a9a4-a856-49a3-a5db-fed5ad314b64'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.920344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] The token cache is being invalidate because project c2d5301114244be2a74210484c6536ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:01:27.920629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.921237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-eb68a9a4-a856-49a3-a5db-fed5ad314b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:27.922808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1192/2390] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:01:27 2022] DELETE /identity/v3/projects/c2d5301114244be2a74210484c6536ef => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:27.928098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:27.928416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:27.928698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:27.947161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:27.948440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:28.009632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1199/2391] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:28.017827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.018478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.018758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.038390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:28.039606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:28.048464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1193/2392] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:28.052784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.062688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e02c9-d8d8-4d0c-bccf-90c52853d525 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.067797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.068488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.068776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.069053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.069785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:28.070095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.071708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.084323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1200/2393] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:28.088509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.097882 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17084bbf-08ba-47b0-b5ef-3178b476b638 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.103055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.103704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.104021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.104294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.104978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.106893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.135662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1194/2394] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:01:28 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:28.140019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.150085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e2e57-3bf3-4559-a3c2-4b663339e48e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.154193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:28.214010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.214688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.214967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.215246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.215967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.217967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.242874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1201/2395] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:01:28 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:28.247323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.258403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb55b7d-3da5-489c-94b5-4c712520892f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.264569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.265244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.265522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.265791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.266517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.268504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.279737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1195/2396] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:28.284026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.294759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f61c8-ac13-4835-a554-e8adc93ca1b2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.300341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.300934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.301220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.301494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.302212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.303919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9c427-5350-4e00-867d-768b3d8f0fb7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.314913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1202/2397] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:28.319342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.329556 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eb990-3c29-4512-ae65-f58f874c0c77 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.334654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.335537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.335812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.336112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] PATH_INFO: `/v3/projects/4226e0702fef42dc9b88fb02e9b3c76a/users/bd2a363ad57045ed978ba3592281a5d7/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.353904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4226e0702fef42dc9b88fb02e9b3c76a, user_id=bd2a363ad57045ed978ba3592281a5d7, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.358066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.374224 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1196/2398] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:28 2022] PUT /identity/v3/projects/4226e0702fef42dc9b88fb02e9b3c76a/users/bd2a363ad57045ed978ba3592281a5d7/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:28.385425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.385715 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.385998 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.425691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] MFA Rules not processed for user `bd2a363ad57045ed978ba3592281a5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:28.426931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:28.493121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1203/2399] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:28.507675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:28.510774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:28.546785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:28.600272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2d72d-8c03-4d88-9d2c-2a901d3730ba admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:28.606592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:28.607248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:28.607532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:28.607809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:28.616399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:28.617998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:28.628675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1197/2400] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:33.505345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:33.506975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:33.568880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ea413-9e46-4cbe-a877-a9f9f666fae9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:33.572893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:33.573529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:33.573806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:33.574077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:33.576094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:33.590684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:33.592170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:33.603897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1204/2401] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:34.831115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:34.832862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:34.846193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc174-573d-4706-88d1-dd283a7e69fd service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:34.850973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:34.851727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:34.852045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:34.852314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:34.854338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:34.868576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:34.870012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c759231-2ff1-44ce-8b15-2092963361f3 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:34.881688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1198/2402] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:36.137784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:36.156117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:36.201695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a56b3-85aa-45ba-9cca-adc240a28a28 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:36.207548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:36.208462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.208736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.209015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] PATH_INFO: `/v3/users/bd2a363ad57045ed978ba3592281a5d7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:36.210309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2a363ad57045ed978ba3592281a5d7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:36.212286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:36.270923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2a363ad57045ed978ba3592281a5d7'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.287215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2a363ad57045ed978ba3592281a5d7'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.288207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2a363ad57045ed978ba3592281a5d7'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.295023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2a363ad57045ed978ba3592281a5d7'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.304346 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2a363ad57045ed978ba3592281a5d7'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.310697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1205/2403] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:01:36 2022] DELETE /identity/v3/users/bd2a363ad57045ed978ba3592281a5d7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:36.315530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.316035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.316149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:36.333521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:36.334721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:36.397393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1199/2404] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:36.407542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:36.409276 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:36.468597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa7e23-46d0-4407-bc24-28090af940b7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:36.473198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:36.473840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.474118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.474387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:36.484704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:36.486127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11089b-6b45-4243-bfad-11f361ddd198 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:36.495654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1206/2405] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:36.767862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:36.769577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:36.784735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523e00-527d-487f-b471-0c4ea4c6b6fa None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:36.790736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:36.791692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.792836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.793583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] PATH_INFO: `/v3/projects/4226e0702fef42dc9b88fb02e9b3c76a` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:36.795459 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4226e0702fef42dc9b88fb02e9b3c76a)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:36.797898 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:36.853488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4226e0702fef42dc9b88fb02e9b3c76a', 'request_id': 'req-8838432e-649c-4d99-a6f4-e5b666f8179b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.860037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4226e0702fef42dc9b88fb02e9b3c76a', 'request_id': 'req-8838432e-649c-4d99-a6f4-e5b666f8179b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.868665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4226e0702fef42dc9b88fb02e9b3c76a', 'request_id': 'req-8838432e-649c-4d99-a6f4-e5b666f8179b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.883983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] The token cache is being invalidate because project 4226e0702fef42dc9b88fb02e9b3c76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:01:36.884527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.885177 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8838432e-649c-4d99-a6f4-e5b666f8179b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:36.886789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1200/2406] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:01:36 2022] DELETE /identity/v3/projects/4226e0702fef42dc9b88fb02e9b3c76a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:36.891896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.892198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.892480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:36.910951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:36.912188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:36.947185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:36.974717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1207/2407] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:36.982606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:36.982891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:36.983166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.002017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:37.003248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.011983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1201/2408] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:37.018234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.029324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffbd2-05d6-440c-aa60-8e370c9a5169 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.034991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.035753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.035919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.036239 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.036956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:37.037269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.038946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.051243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1208/2409] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:37.055528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.064961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccab2b-1af8-4d99-929d-cd249fc4367a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.070966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.071637 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.071911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.072208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.072902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.074866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.089644 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:37.106106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1202/2410] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:37.115548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.125919 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ab97f-73df-46f4-a81e-e384e5ae650b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.186624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.187351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.187597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.187891 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.188583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.190534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.214049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1209/2411] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/users => generated 279 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:37.218087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.228853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc528-217d-458d-8a6a-3e89e8d72204 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.233617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.234272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.234546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.234815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.235549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.237347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.248042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1203/2412] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:01:37.252135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.264136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc7587-37e6-452f-a0ef-b7c6c80bf331 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.269710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.270618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.270975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.271427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.272373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.274089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34b90-60ae-4b9b-9a1e-a3019d36aaa5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.285749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1210/2413] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:37.292684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.302866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae237bd-3631-4c99-ad21-a0869b54db94 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.307570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.308481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.308758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.309055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] PATH_INFO: `/v3/projects/86ae1b420878439694007eb2d2743991/users/591854d8d56947b48aaf069e3fbe0197/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.325441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86ae1b420878439694007eb2d2743991, user_id=591854d8d56947b48aaf069e3fbe0197, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.328657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.343595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1204/2414] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:37 2022] PUT /identity/v3/projects/86ae1b420878439694007eb2d2743991/users/591854d8d56947b48aaf069e3fbe0197/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:37.354073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.354362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.354640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.385984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] MFA Rules not processed for user `591854d8d56947b48aaf069e3fbe0197`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:37.387214 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.453945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1211/2415] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:37.459169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.528014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47374-51f6-4afd-91e9-903026ebdc51 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.533147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.533789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.534068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.534337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.535014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.536941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.564055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1205/2416] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:37.568272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.579381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104187b8-a6ed-4b1b-bc4b-d4d37a2acb52 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.614091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:37.640927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.641594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.641870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.642143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.642868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.644813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.668943 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1212/2417] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:37.674462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.684851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618c518-8c95-4d06-a32d-cfd99b05f081 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.689880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.690527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.690805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.691101 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.691789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.693439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.703732 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1206/2418] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:37.707960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.717929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9f2ad-0223-448c-8b29-0a00b2a6c159 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.722962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.723864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.724124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.724395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] PATH_INFO: `/v3/projects/3b517647e2974c36ac357cfec958e4a2/users/a5f09fe926ce40548b3427f56e1137ea/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.740908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b517647e2974c36ac357cfec958e4a2, user_id=a5f09fe926ce40548b3427f56e1137ea, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.743558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.758187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1213/2419] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:37 2022] PUT /identity/v3/projects/3b517647e2974c36ac357cfec958e4a2/users/a5f09fe926ce40548b3427f56e1137ea/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:37.762236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.773291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dedad-7b35-4777-8645-278695baf004 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.835536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.836368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.836640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.837006 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.838098 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.840682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5a1a-6c8a-476b-a703-3b962db1ead2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.852819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1207/2420] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:37.857144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:37.868682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd79816-7a19-4aaa-a536-afb97d552973 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:37.873950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:37.874871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.875159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.875435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] PATH_INFO: `/v3/projects/3b517647e2974c36ac357cfec958e4a2/users/a5f09fe926ce40548b3427f56e1137ea/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.877666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b517647e2974c36ac357cfec958e4a2, user_id=a5f09fe926ce40548b3427f56e1137ea, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:37.880396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:37.890867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1214/2421] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:37 2022] PUT /identity/v3/projects/3b517647e2974c36ac357cfec958e4a2/users/a5f09fe926ce40548b3427f56e1137ea/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:37.903230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:37.903512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:37.903790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:37.931856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] MFA Rules not processed for user `a5f09fe926ce40548b3427f56e1137ea`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:37.933087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:37.951083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:38.005343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1208/2422] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:37 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:38.017079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:38.019109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:38.087066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e898-935d-405f-8a11-0857db258bdc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:38.091807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:38.092513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:38.092795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:38.093076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:38.103558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:38.105106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:38.115443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1215/2423] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:38 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:38.196906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:38.208013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1243913-4aeb-4174-b84f-64782779c228 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:38.214857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:38.215737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:38.216287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:38.216617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:38.281215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:38.284175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:38.295642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1209/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:38 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:48.110500 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:48.291678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=49 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:50.182310 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:50.184997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:50.271612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f15b02-764b-48d6-a338-e2d3b98c65a7 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:50.278030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:50.278699 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:50.278974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:50.279252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:50.281188 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:50.348258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:50.349901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee3064-9bbb-4e62-b579-a1fc2d827802 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:50.359149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1216/2425] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:53.308962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:53.375416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fc36b-f287-49b9-b874-7dac4acefd9f service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:53.380679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:53.381562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.381869 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.382137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] PATH_INFO: `/v3/users/591854d8d56947b48aaf069e3fbe0197` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.383305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591854d8d56947b48aaf069e3fbe0197)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:53.385264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:53.435218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591854d8d56947b48aaf069e3fbe0197'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:53.448119 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591854d8d56947b48aaf069e3fbe0197'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:53.448789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591854d8d56947b48aaf069e3fbe0197'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:53.455517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591854d8d56947b48aaf069e3fbe0197'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:53.464296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591854d8d56947b48aaf069e3fbe0197'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:53.470537 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1210/2426] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:01:53 2022] DELETE /identity/v3/users/591854d8d56947b48aaf069e3fbe0197 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:53.475356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be0806-aa63-40bd-a96b-7669cb5bd39c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.475924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be0806-aa63-40bd-a96b-7669cb5bd39c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.475924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be0806-aa63-40bd-a96b-7669cb5bd39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.493168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c5be0806-aa63-40bd-a96b-7669cb5bd39c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:53.494366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c5be0806-aa63-40bd-a96b-7669cb5bd39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:53.554573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1217/2427] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:53.564553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:53.566148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:53.628243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bab160-0a6a-4b5d-ae56-b29195ef8e65 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:53.634200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:53.634862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.635112 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.635378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.645415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:53.646864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:53.657181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1211/2428] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:53.851759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:53.852071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:53.852355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:53.871599 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:53.872842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:53.910778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:53.944393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1218/2429] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:53.990890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:53.992614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:53.999522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.001114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:54.008977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379dfa4-7b2d-45e4-8db8-9966c5c2a0e5 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.028899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.030179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.030477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.030649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] PATH_INFO: `/v3/projects/86ae1b420878439694007eb2d2743991` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.031945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86ae1b420878439694007eb2d2743991)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.034089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.058927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.088795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abef21c-8ce0-401a-83ad-b1d05c1cc0a9 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.097067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ae1b420878439694007eb2d2743991', 'request_id': 'req-5f18e4dd-1670-4c64-83a7-71948f761857'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.103361 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ae1b420878439694007eb2d2743991', 'request_id': 'req-5f18e4dd-1670-4c64-83a7-71948f761857'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.109048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ae1b420878439694007eb2d2743991', 'request_id': 'req-5f18e4dd-1670-4c64-83a7-71948f761857'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.121992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] The token cache is being invalidate because project 86ae1b420878439694007eb2d2743991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:01:54.122280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.122896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.124468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1212/2430] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:01:53 2022] DELETE /identity/v3/projects/86ae1b420878439694007eb2d2743991 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:01:54.128512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.129893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:54.148539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.149198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.149478 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.149757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.151952 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:54.197331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18e4dd-1670-4c64-83a7-71948f761857 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.202144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.203014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.203292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.203562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] PATH_INFO: `/v3/users/a5f09fe926ce40548b3427f56e1137ea` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.204802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f09fe926ce40548b3427f56e1137ea)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.206786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.211692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.213447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.221905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1219/2431] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:01:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 11:01:54.278988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f09fe926ce40548b3427f56e1137ea'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.292695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f09fe926ce40548b3427f56e1137ea'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.293860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f09fe926ce40548b3427f56e1137ea'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.302786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f09fe926ce40548b3427f56e1137ea'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.311649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e67d5a4a-2ede-4f64-883b-8ccf85556518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f09fe926ce40548b3427f56e1137ea'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.319917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1213/2432] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:01:54 2022] DELETE /identity/v3/users/a5f09fe926ce40548b3427f56e1137ea => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:01:54.646380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:54.659704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cd24a-5a59-4457-a092-c017269ca29f service swift] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:54.692779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:54.749564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:54.750445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.750722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.751142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] PATH_INFO: `/v3/projects/3b517647e2974c36ac357cfec958e4a2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.752414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b517647e2974c36ac357cfec958e4a2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:54.754608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:54.836674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b517647e2974c36ac357cfec958e4a2', 'request_id': 'req-70e9eb6f-0ddb-477a-a977-a976b5da2784'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.845433 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b517647e2974c36ac357cfec958e4a2', 'request_id': 'req-70e9eb6f-0ddb-477a-a977-a976b5da2784'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.855385 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b517647e2974c36ac357cfec958e4a2', 'request_id': 'req-70e9eb6f-0ddb-477a-a977-a976b5da2784'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.886716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] The token cache is being invalidate because project 3b517647e2974c36ac357cfec958e4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:01:54.887014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.889963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-70e9eb6f-0ddb-477a-a977-a976b5da2784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:01:54.898719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:54.899054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:54.899264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:54.899736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1220/2433] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:01:54 2022] DELETE /identity/v3/projects/3b517647e2974c36ac357cfec958e4a2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:54.923273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:54.924839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:54.959767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.010137 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1214/2434] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:55.018136 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.018536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.018945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.038033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:55.039415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:55.047688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1221/2435] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:55.052331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.064760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d336-ee68-4a47-b78c-7ef862263ddf None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.069830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.070611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.071014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.071362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.072193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:01:55.072573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.074254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.092950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1215/2436] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:01:55.094571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.114575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1995a1-4b57-4112-9340-aea0c7e0e8de None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.119861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.120695 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.121086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.121458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.122308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.124447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.165491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1222/2437] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/projects => generated 409 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:55.169822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.190567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26485b-b1fe-4331-95eb-7c0ccb19f372 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.268659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.269914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.270316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.270682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.271574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.273920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.304568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1216/2438] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/users => generated 305 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:55.308462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.321652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc8f1d-4cc3-4e61-b1f8-b4bbddd1c8e6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.326872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.327908 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.328289 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.328675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.329606 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.331763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.355590 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1223/2439] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:55.356782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.368004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88927127-f547-4ad9-bcf0-587e20f638f7 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.372748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.373435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.373719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.374008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.374763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.376437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202d69-d587-47d0-abe8-c3f25d930ef0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.391028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1217/2440] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:55.392648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.407491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69882d72-aa51-4075-a5d7-f5d7091e72ac admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.413336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.414341 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.414720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.415099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] PATH_INFO: `/v3/projects/465b88d702a54e7e9579dd2513136c69/users/35930d5b03c04689824fb669cea93609/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.434451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465b88d702a54e7e9579dd2513136c69, user_id=35930d5b03c04689824fb669cea93609, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.437337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.463362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1224/2441] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:55 2022] PUT /identity/v3/projects/465b88d702a54e7e9579dd2513136c69/users/35930d5b03c04689824fb669cea93609/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:01:55.466653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.467148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.467530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.503500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] MFA Rules not processed for user `35930d5b03c04689824fb669cea93609`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:55.505195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:55.596639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1218/2442] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:01:55.600533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.662121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.673846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a0b2-aa83-4f3c-8bcb-c641dbfd0e92 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.680142 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.681101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.681746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.682040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.682762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.684763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.715776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1225/2443] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/projects => generated 409 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:55.719919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.732636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.734472 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:55.744624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8ecc-ed13-4489-83d7-225f0a80ea6a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.769037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:55.832150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.833103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.833531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.833953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.834959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.837110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.882532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1219/2444] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:01:55 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:01:55.886549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.905587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62a7a5-38da-4117-bf07-a3d096968fe5 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.908247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d9289-7329-430e-85fe-1f33a22a24f7 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:55.919944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.921084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.921516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.922075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.925483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:55.936517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:55.938045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:55.938477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:55.938899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:55.947643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:55.950889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:55.974836 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1220/2445] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:01:55.978992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:55.995713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303273-5f6a-476a-9873-5894ed5325f8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:56.002749 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:56.004170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.005409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.005802 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] PATH_INFO: `/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d/users/fc31cd36d2be4eb6a56d3fc09884847e/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.024351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.026240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.028906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0b7bc2c33d484b8ad4acfe71e4e74d, user_id=fc31cd36d2be4eb6a56d3fc09884847e, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.031942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.037230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1226/2446] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:56.056589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1221/2447] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:55 2022] PUT /identity/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d/users/fc31cd36d2be4eb6a56d3fc09884847e/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:01:56.060429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:56.075014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4181a-04b8-45ef-9db2-ee01a81328f2 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:56.146718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:56.147479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.147834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.148237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.149043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.150742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a70b4-bce1-4b19-9d38-aac587b2659d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.170900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1227/2448] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:01:56.172059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:56.186014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c3e35-2a14-4f99-bf0e-88e8f67ad06b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:56.190742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:56.191682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.192567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.192567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] PATH_INFO: `/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d/users/fc31cd36d2be4eb6a56d3fc09884847e/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.196052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0b7bc2c33d484b8ad4acfe71e4e74d, user_id=fc31cd36d2be4eb6a56d3fc09884847e, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.199860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.210894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1222/2449] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:01:56 2022] PUT /identity/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d/users/fc31cd36d2be4eb6a56d3fc09884847e/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:01:56.221415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.221792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.222386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.245765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:01:56.261386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] MFA Rules not processed for user `fc31cd36d2be4eb6a56d3fc09884847e`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:01:56.263009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:56.370512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1228/2450] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:01:56.378764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:56.380591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:01:56.458507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dcd5d-f610-4f53-97b6-56a960011175 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:56.462972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:56.463734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.463998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.464170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.485983 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.487513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.503182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1223/2451] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:01:56.704072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:01:56.719984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81581b-05ab-43a1-abc1-443116adc1d4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:01:56.723786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:01:56.724482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:01:56.724764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:01:56.725040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:01:56.802774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:01:56.805298 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:01:56.822094 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1229/2452] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:04.581626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:04.583380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:04.601223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:04.670902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205e848-b243-4b04-b078-91115899cab9 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:04.672061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5b702-ab62-4a8c-9238-f974bde4381c service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:04.674872 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:04.675673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:04.676131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:04.676461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:04.678087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:04.678863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:04.678996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:04.679266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:04.679622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] PATH_INFO: `/v3/users/35930d5b03c04689824fb669cea93609` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:04.681031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35930d5b03c04689824fb669cea93609)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:04.683049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:04.691851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:04.693553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262e077-6b43-4bcb-80ad-d7911fcc42e5 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:04.703634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1224/2453] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:04.726965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:04.736692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35930d5b03c04689824fb669cea93609'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:04.749251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35930d5b03c04689824fb669cea93609'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:04.749964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35930d5b03c04689824fb669cea93609'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:04.757337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35930d5b03c04689824fb669cea93609'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:04.767849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35930d5b03c04689824fb669cea93609'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:04.774595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1230/2454] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:04 2022] DELETE /identity/v3/users/35930d5b03c04689824fb669cea93609 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:02:04.779499 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:04.779912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:04.780194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:04.792469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:04.799349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:04.801622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:04.871109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1225/2455] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:04.878612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:04.880728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:04.952769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5d9c8-86b4-4b6e-b164-77b2d6636056 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:04.957261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:04.958466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:04.958743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:04.959015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:04.969242 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:04.970718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:04.987384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1231/2456] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:05.402152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:05.404823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:05.419411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173b309-b6ed-4ba9-869c-df5f18b7146f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:05.425051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:05.425962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:05.426251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:05.426515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] PATH_INFO: `/v3/projects/465b88d702a54e7e9579dd2513136c69` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:05.427765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=465b88d702a54e7e9579dd2513136c69)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:05.429722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:05.481504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465b88d702a54e7e9579dd2513136c69', 'request_id': 'req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.487864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465b88d702a54e7e9579dd2513136c69', 'request_id': 'req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.493610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465b88d702a54e7e9579dd2513136c69', 'request_id': 'req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.507535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] The token cache is being invalidate because project 465b88d702a54e7e9579dd2513136c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:05.507821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.508465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.510046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1226/2457] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:05 2022] DELETE /identity/v3/projects/465b88d702a54e7e9579dd2513136c69 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:05.514260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:05.524415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b820c5-355a-4f4b-88e5-bc125e02d983 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:05.582394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:05.583456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:05.583950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:05.584229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] PATH_INFO: `/v3/users/fc31cd36d2be4eb6a56d3fc09884847e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:05.585611 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc31cd36d2be4eb6a56d3fc09884847e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:05.587685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:05.647584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc31cd36d2be4eb6a56d3fc09884847e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.662423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:05.666275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc31cd36d2be4eb6a56d3fc09884847e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.667102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc31cd36d2be4eb6a56d3fc09884847e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.675758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc31cd36d2be4eb6a56d3fc09884847e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.684614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-502be87f-0914-4051-b01e-9ac0fe27f708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc31cd36d2be4eb6a56d3fc09884847e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:05.690999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1232/2458] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:05 2022] DELETE /identity/v3/users/fc31cd36d2be4eb6a56d3fc09884847e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:05.998226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:05.999755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:06.018090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6ca37-33b2-4fe2-8e87-7e71745ba90e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.082539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.083420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.083702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.084017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] PATH_INFO: `/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.085255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0b7bc2c33d484b8ad4acfe71e4e74d)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.087220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.121986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:06.136416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0b7bc2c33d484b8ad4acfe71e4e74d', 'request_id': 'req-6ef22dbe-ce69-4f44-a295-0e407c02eaee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:06.143191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0b7bc2c33d484b8ad4acfe71e4e74d', 'request_id': 'req-6ef22dbe-ce69-4f44-a295-0e407c02eaee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:06.149720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0b7bc2c33d484b8ad4acfe71e4e74d', 'request_id': 'req-6ef22dbe-ce69-4f44-a295-0e407c02eaee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:06.166075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] The token cache is being invalidate because project 4b0b7bc2c33d484b8ad4acfe71e4e74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:06.166258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:06.167197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6ef22dbe-ce69-4f44-a295-0e407c02eaee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:06.168784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1227/2459] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:05 2022] DELETE /identity/v3/projects/4b0b7bc2c33d484b8ad4acfe71e4e74d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:06.174117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.174424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.174705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.195259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:06.196496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:06.259457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1233/2460] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:06.267303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.268238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.268579 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.288437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:06.289635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:06.299344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1228/2461] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:06.304730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.314615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eea824-3b27-4f2c-81e4-729774867a90 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.319697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.320377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.320650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.320918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.321637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:06.321939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.323531 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.336047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1234/2462] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:06.340187 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.350002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b41ab-8198-4188-b439-dc4d5b78d660 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.354851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.355476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.355758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.356061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.356744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.358632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.386623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1229/2463] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:06.390903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.401459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8c0a-0d4c-4421-8a5a-50d7b210e485 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.459548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.460147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.460463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.460729 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.461425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.463575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.490382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1235/2464] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:02:06.493071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.503514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056ee6-0ba5-4652-b093-c4a6a30758e3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.508377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.509043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.509325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.509601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.510331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.511980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.522481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1230/2465] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:06.526571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.537542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef41d2-2275-4dba-819b-0ef4936713bb admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.542720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.543377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.543680 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.543950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.544670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.546299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90718271-df10-4de6-a9fd-f6d4f785d689 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.557296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1236/2466] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:06.561794 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.573096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1668f-2fe3-4a9f-8fc4-22859f989ce0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.578141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.579107 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.579388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.579659 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] PATH_INFO: `/v3/projects/27fcf3fdfe8544ab82ae2722bfbac2e3/users/c6ca0090943a487bb31c87009c98a342/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.598362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fcf3fdfe8544ab82ae2722bfbac2e3, user_id=c6ca0090943a487bb31c87009c98a342, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.601162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.617225 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1231/2467] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:06 2022] PUT /identity/v3/projects/27fcf3fdfe8544ab82ae2722bfbac2e3/users/c6ca0090943a487bb31c87009c98a342/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:06.628538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.628828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.629181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.658117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] MFA Rules not processed for user `c6ca0090943a487bb31c87009c98a342`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:06.659362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:06.692548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:06.734625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1237/2468] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:06.739512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.806249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411d174-fc80-48c2-aa3b-ec6ebb2f306e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.811516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.812222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.812511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.812791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.813487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.815418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:06.843933 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1232/2469] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:06.847957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:06.861355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee0fc5-c2ac-4373-9ec2-6b367183950b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:06.954056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:06.954765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:06.956709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:06.956709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:06.956709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:06.959007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.000901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1238/2470] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:02:06 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:07.004617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.018362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768bfd8-20dc-4a4d-8ac8-4dd243925695 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.023914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.024687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.024970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.025295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.026151 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.028305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.048427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1233/2471] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:07.051995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.062899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082ea43-c5f8-4fd9-886c-57b51be74053 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.067683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.068610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.068892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.069198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] PATH_INFO: `/v3/projects/96c47f553e2b47e8adf7ca8247314147/users/5414e3c34d08449cb5762bc0e0b957e9/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.085994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c47f553e2b47e8adf7ca8247314147, user_id=5414e3c34d08449cb5762bc0e0b957e9, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.088390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.102879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1239/2472] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:07 2022] PUT /identity/v3/projects/96c47f553e2b47e8adf7ca8247314147/users/5414e3c34d08449cb5762bc0e0b957e9/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:07.106867 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.117213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240175d1-f1c9-4dc8-b99b-89311fc29961 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.147248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:07.180048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.180714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.180996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.181272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.182000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.183632 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fce00-618f-4c7c-8656-dc666239782d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.194153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1234/2473] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:07.199941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.210775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6df0a-c5f8-4610-afcd-3062f440ee91 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.215602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.216585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.216866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.217147 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] PATH_INFO: `/v3/projects/96c47f553e2b47e8adf7ca8247314147/users/5414e3c34d08449cb5762bc0e0b957e9/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.219329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c47f553e2b47e8adf7ca8247314147, user_id=5414e3c34d08449cb5762bc0e0b957e9, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.222104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.238265 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1240/2474] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:07 2022] PUT /identity/v3/projects/96c47f553e2b47e8adf7ca8247314147/users/5414e3c34d08449cb5762bc0e0b957e9/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:02:07.248485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.248842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.249045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.321535 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] MFA Rules not processed for user `5414e3c34d08449cb5762bc0e0b957e9`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:07.322585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:07.440603 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1235/2475] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:07.451946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:07.455636 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:07.526668 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd068542-965f-467b-9ed5-830a393c62ba admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:07.531231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:07.532015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:07.532367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:07.532873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:07.543764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:07.545343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f605f-8c75-4db9-b9ec-76f48dd94f4c service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:07.562460 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1241/2476] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:08.123609 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.201602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3cabf-e208-4958-b5fe-3d08dcc69ba3 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.206693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.207923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.208652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.209030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] PATH_INFO: `/v3/users/c6ca0090943a487bb31c87009c98a342` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.210583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ca0090943a487bb31c87009c98a342)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.212968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.264000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ca0090943a487bb31c87009c98a342'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.269395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:08.278512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ca0090943a487bb31c87009c98a342'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.279394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ca0090943a487bb31c87009c98a342'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.286658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ca0090943a487bb31c87009c98a342'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.295133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ca0090943a487bb31c87009c98a342'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.301474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1236/2477] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:08 2022] DELETE /identity/v3/users/c6ca0090943a487bb31c87009c98a342 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:08.307470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab9ab-d924-4236-a107-47389214bd98 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.308457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab9ab-d924-4236-a107-47389214bd98 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.308811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab9ab-d924-4236-a107-47389214bd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.327573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4fdab9ab-d924-4236-a107-47389214bd98 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:08.328914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4fdab9ab-d924-4236-a107-47389214bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.353283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:08.396124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1242/2478] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:08.406131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.407628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.467697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f907e85-04c4-4955-b812-baf794a73216 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.471941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.472592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.472870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.473404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.483306 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.484790 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6a261-3a6e-4cf2-a4e5-08e36c4cbd19 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.496666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1237/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:08.532559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.533206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.533705 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] PATH_INFO: `/` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.535575 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1243/2480] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 11:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:08.542585 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8809f24-d8f6-409a-8551-13f80066644b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.542902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8809f24-d8f6-409a-8551-13f80066644b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.543150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8809f24-d8f6-409a-8551-13f80066644b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.566502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c8809f24-d8f6-409a-8551-13f80066644b None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:08.568270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c8809f24-d8f6-409a-8551-13f80066644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.646682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1238/2481] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:08.659033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.661229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.725029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb022434-b998-4fca-89ca-ccb7f9af4381 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.730378 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.732128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.732944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.733295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.746026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.747382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7e29f-8256-4a80-a40b-969e70a01fb4 service glance] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.759585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1244/2482] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:08.836838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.837439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.837771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.839375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1239/2483] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 11:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 11:02:08.845266 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.845623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.845854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.875904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] MFA Rules not processed for user `69a3d1b0e64049d28adc1a4f9ea10337`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:08.877231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.882765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.886666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.893564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1245/2484] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:08.912178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:08.914322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:08.915676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617af6e1-5838-45b9-9b64-382384ac1f1b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.920717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.922086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.922464 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.922904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] PATH_INFO: `/v3/projects/27fcf3fdfe8544ab82ae2722bfbac2e3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.924759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fcf3fdfe8544ab82ae2722bfbac2e3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.927516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.928473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61c3f-5d5d-4aa1-90d2-7525c7238dda None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:08.932427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:08.933559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:08.933844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:08.934119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:08.943218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:08.944731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:08.954379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1246/2485] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:02:08.987163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fcf3fdfe8544ab82ae2722bfbac2e3', 'request_id': 'req-1e435281-c36c-4113-bae6-f5a44d86e9e3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.993668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fcf3fdfe8544ab82ae2722bfbac2e3', 'request_id': 'req-1e435281-c36c-4113-bae6-f5a44d86e9e3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:08.999593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fcf3fdfe8544ab82ae2722bfbac2e3', 'request_id': 'req-1e435281-c36c-4113-bae6-f5a44d86e9e3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.001421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:09.015586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] The token cache is being invalidate because project 27fcf3fdfe8544ab82ae2722bfbac2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:09.015914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.016785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-1e435281-c36c-4113-bae6-f5a44d86e9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.018857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1240/2486] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:08 2022] DELETE /identity/v3/projects/27fcf3fdfe8544ab82ae2722bfbac2e3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:02:09.023042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.035936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165352d8-f0e8-4805-a16e-34344a8518d8 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:09.043271 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa15b3-30b3-4b84-bd7c-ca995cdc5e66 None None] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.043487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa15b3-30b3-4b84-bd7c-ca995cdc5e66 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.043776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa15b3-30b3-4b84-bd7c-ca995cdc5e66 None None] PATH_INFO: `/` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.045342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1241/2487] 10.222.0.27 () {54 vars in 921 bytes} [Thu Jul 14 11:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 11:02:09.050681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.050965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.051290 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.070666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] MFA Rules not processed for user `d08cdb2d283143aa83b38bb3b04aee8c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:09.071897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.099082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:09.100085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.100349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.100683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] PATH_INFO: `/v3/users/5414e3c34d08449cb5762bc0e0b957e9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.102044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5414e3c34d08449cb5762bc0e0b957e9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:09.104233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:09.149693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1242/2488] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Jul 14 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:09.158932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5414e3c34d08449cb5762bc0e0b957e9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.162861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.164216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:09.164700 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.175946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5414e3c34d08449cb5762bc0e0b957e9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.176768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5414e3c34d08449cb5762bc0e0b957e9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.185622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5414e3c34d08449cb5762bc0e0b957e9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.194887 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5414e3c34d08449cb5762bc0e0b957e9'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.202734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1247/2489] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:09 2022] DELETE /identity/v3/users/5414e3c34d08449cb5762bc0e0b957e9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:09.239580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe538b4-a518-42f4-a3eb-0e8bfa40feb0 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:09.295385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:09.296002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.296425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.296710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.298571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.348471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:09.373294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:09.374833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:09.390396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1243/2490] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:02:09.488765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.491028 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.572909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72bf6-1a8c-4f3c-97d2-52fd50669d31 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:09.579492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:09.582066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.582358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.582833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.584869 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.590817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.592666 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.600812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:09.603001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4d8a3-999d-4773-aa20-c732e66e1016 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:09.615063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1248/2491] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:09.679682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b63999-9251-466a-812f-5de6ebad4128 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:09.684398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:09.685266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.685574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.685850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] PATH_INFO: `/v3/projects/96c47f553e2b47e8adf7ca8247314147` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.687031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c47f553e2b47e8adf7ca8247314147)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:09.688992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:09.739014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c47f553e2b47e8adf7ca8247314147', 'request_id': 'req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.745519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c47f553e2b47e8adf7ca8247314147', 'request_id': 'req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.751451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c47f553e2b47e8adf7ca8247314147', 'request_id': 'req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.765635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] The token cache is being invalidate because project 96c47f553e2b47e8adf7ca8247314147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:09.765927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.766596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b5c5a64a-99f7-4e6c-adc3-53c64675ee4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:09.768175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1244/2492] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:09 2022] DELETE /identity/v3/projects/96c47f553e2b47e8adf7ca8247314147 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:02:09.773222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.773594 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.773906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.794975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:09.796215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.877197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1249/2493] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:09.886059 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.886350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.886656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.909992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:09.911947 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:09.929916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1245/2494] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:09.934203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.952425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56ad63-5477-4c54-822c-db2357d37cbe None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:09.958802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:09.959505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:09.959807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:09.960097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:09.960846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:09.961413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:09.963303 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:09.978247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1250/2495] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:09.991256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:09.996357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbdd8e-4abd-41a5-b554-f5da5406c00c None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.002179 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.004171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.004998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.005343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.006132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.008961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.041003 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1246/2496] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:02:09 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:10.045323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.055700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5480f-b77c-4f91-9b1d-a79073650899 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.102187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:10.118063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.118827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.119137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.119471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.120244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.122236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.144683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1251/2497] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:02:10 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:10.149217 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.160532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee77368-c7d8-4ef0-b464-1e142bd9a699 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.167053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.167755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.168146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.168421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.169150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.170770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.182005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1247/2498] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:10.186189 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.196860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81250cb-81ab-473c-84e8-e55b1147ff15 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.202025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.202707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.202994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.203273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.204092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.205738 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35993aea-11b7-4830-96d8-1c0815be967d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.216083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1252/2499] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:10.220196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.231397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4b777-28cb-434d-969b-c568b573f4de admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.237189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.238102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.238381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.238653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] PATH_INFO: `/v3/projects/106445e873fc4278b78aa58286571891/users/563151b169cc48a483c24eb2c3b245b1/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.257981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=106445e873fc4278b78aa58286571891, user_id=563151b169cc48a483c24eb2c3b245b1, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.262149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.280049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1248/2500] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:10 2022] PUT /identity/v3/projects/106445e873fc4278b78aa58286571891/users/563151b169cc48a483c24eb2c3b245b1/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:10.291710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c36f2-3a87-44c8-9acd-949a83516c0a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.292049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c36f2-3a87-44c8-9acd-949a83516c0a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.292343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c36f2-3a87-44c8-9acd-949a83516c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.321022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-183c36f2-3a87-44c8-9acd-949a83516c0a None None] MFA Rules not processed for user `563151b169cc48a483c24eb2c3b245b1`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:10.322247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-183c36f2-3a87-44c8-9acd-949a83516c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:10.394181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1253/2501] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:10.406407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:10.408379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:10.416268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:10.469522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd53605-e6f6-41cc-8a92-bfba0bed77bb admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:10.473557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:10.474329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:10.474679 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:10.474962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:10.484273 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:10.485858 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:10.497320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1249/2502] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:11.439450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:11.440375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:11.440835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:11.463567 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:11.464824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:11.538159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1254/2503] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:11.550746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:11.552371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:11.631077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56628-c822-415c-803d-e0588b036c93 service glance] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:11.638975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:11.639768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:11.640284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:11.640661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:11.652280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:11.654219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0a4d-86df-4870-9016-cd75af1c36e2 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:11.664938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1250/2504] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:02:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:02:21.528800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:21.660624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=56 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:24.958130 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:24.959670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:25.039608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a6fd3-c315-46fd-9ce6-533940d1c8c2 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:25.045670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:25.046322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:25.046610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:25.046883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:25.048892 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:25.061719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:25.063329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:25.074294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1255/2505] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:35.069389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=34 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:38.911244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:38.911791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:38.911791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:38.935989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:38.939453 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:38.948583 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1251/2506] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 11:02:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:38.960958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:38.973893 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bc247-a7c6-47f4-9055-d9520a3facc8 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:38.983313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:38.984532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:38.985963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:38.985963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:38.996328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:38.999789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a8ab4-5619-4314-bf4d-08570693c527 service swift] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:39.009622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1256/2507] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:02:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:02:44.616070 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:44.682945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bc0d4-50a1-4877-890d-856a5fe375ee None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:44.687387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:44.688316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:44.688598 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:44.688875 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] PATH_INFO: `/v3/users/563151b169cc48a483c24eb2c3b245b1` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:44.690081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=563151b169cc48a483c24eb2c3b245b1)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:44.692090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:44.741196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563151b169cc48a483c24eb2c3b245b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:44.753372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563151b169cc48a483c24eb2c3b245b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:44.754134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563151b169cc48a483c24eb2c3b245b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:44.761079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563151b169cc48a483c24eb2c3b245b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:44.770103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563151b169cc48a483c24eb2c3b245b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:44.777337 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1252/2508] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:44 2022] DELETE /identity/v3/users/563151b169cc48a483c24eb2c3b245b1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:44.782318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:44.782751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:44.782860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:44.804349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:44.805935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:44.847457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:44.849123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:44.916133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1257/2509] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:44.925879 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:44.927416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:44.964819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691babd-8ca9-4e3f-9f7a-8704dafe6c6a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:44.977910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:44.978633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:44.978920 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:44.979192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:44.982940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:45.007542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.009418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.015152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:45.023289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1253/2510] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:45.099381 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d585900-7a74-434f-9bae-4ed86c3f0329 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.103488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.104413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.104868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.105299 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.121885 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.123789 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f91bb-44f9-4f36-ab28-82ead9b62029 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.143651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1258/2511] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:45.425784 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.428548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:45.443996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d06aaa-a443-4422-9a3b-641a0308cd24 service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.448866 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.449741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.450056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.450330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] PATH_INFO: `/v3/projects/106445e873fc4278b78aa58286571891` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.451506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106445e873fc4278b78aa58286571891)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.453465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.505109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106445e873fc4278b78aa58286571891', 'request_id': 'req-804f25c8-c049-4895-b407-cedb187566d9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:45.512174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106445e873fc4278b78aa58286571891', 'request_id': 'req-804f25c8-c049-4895-b407-cedb187566d9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:45.517902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106445e873fc4278b78aa58286571891', 'request_id': 'req-804f25c8-c049-4895-b407-cedb187566d9'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:45.533975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] The token cache is being invalidate because project 106445e873fc4278b78aa58286571891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:45.534258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:45.534893 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-804f25c8-c049-4895-b407-cedb187566d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:45.536489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1254/2512] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:45 2022] DELETE /identity/v3/projects/106445e873fc4278b78aa58286571891 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:45.541522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.541803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.542073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.559263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:45.560529 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:45.629835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1259/2513] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:45.632923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.633685 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.634063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.661042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:45.665483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:45.677724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1255/2514] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:45.682134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.696794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d214873-c30b-4474-8517-3c4a222abce7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.703343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.704074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.704357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.704628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.705360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:45.705665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.707319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.721080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1260/2515] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:45.725398 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.737387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb48699-ee12-465b-ba8d-539fc64f07f5 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.750153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.750842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.751127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.751405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.752163 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.754278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.785594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1256/2516] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:02:45 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:45.789749 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.800422 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7ae5-999e-4c89-a6d5-0fcaafce3e19 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.814075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:45.870112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.870807 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.871079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.871352 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.872086 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.874068 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.895566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1261/2517] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:02:45 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:02:45.899547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.910414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4aa363-82a6-4ef6-b8d4-9743091572e5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.920395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.921146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.921440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.921717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.922488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.924241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.939500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1257/2518] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:45.941085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.952602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61e58b-74d8-4b9d-9818-208ad299fbf6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.957246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.957917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.958198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.958471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:45.959200 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:45.960948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac8387-73f1-4422-998b-ae664773198d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:45.971718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1262/2519] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:45.975992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:45.987048 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbf726-da68-4fbf-8750-2f91fc918ce6 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:45.991513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:45.992442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:45.992718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:45.993072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] PATH_INFO: `/v3/projects/ad57e412dc1e46c39f1121f3e4ca8ee9/users/75a9903b3597458f8f5af4d06cd07d4c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:46.010642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad57e412dc1e46c39f1121f3e4ca8ee9, user_id=75a9903b3597458f8f5af4d06cd07d4c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:46.013575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:46.029292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1258/2520] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:45 2022] PUT /identity/v3/projects/ad57e412dc1e46c39f1121f3e4ca8ee9/users/75a9903b3597458f8f5af4d06cd07d4c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:46.035382 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6caa-f157-4dcb-8d3d-73f875f361ea None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:46.035671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6caa-f157-4dcb-8d3d-73f875f361ea None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:46.036017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a6caa-f157-4dcb-8d3d-73f875f361ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:46.055408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-443a6caa-f157-4dcb-8d3d-73f875f361ea None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:46.056663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-443a6caa-f157-4dcb-8d3d-73f875f361ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:46.119520 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1263/2521] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:46.129540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:46.131454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:46.190346 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d88cc-7f76-42f5-85d6-aeed2a4056c2 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:46.194733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:46.195380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:46.195660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:46.195942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:46.205989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:46.208343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:46.221740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1259/2522] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:46.962309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb33a-36e7-4b1d-953f-b4c950e82cbf None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:46.962884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb33a-36e7-4b1d-953f-b4c950e82cbf None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:46.962884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb33a-36e7-4b1d-953f-b4c950e82cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:46.996862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f6ecb33a-36e7-4b1d-953f-b4c950e82cbf None None] MFA Rules not processed for user `75a9903b3597458f8f5af4d06cd07d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:46.998065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f6ecb33a-36e7-4b1d-953f-b4c950e82cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:47.093372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1264/2523] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:47.104539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:47.106108 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:47.131193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:47.175776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff460-dba3-422c-b6a9-e98df720df27 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:47.180278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:47.181106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:47.181899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:47.182044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:47.192481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:47.194452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:47.216168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1260/2524] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:48.582148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:48.582711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:48.582711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:48.602155 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] MFA Rules not processed for user `75a9903b3597458f8f5af4d06cd07d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:48.603474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:48.611416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1265/2525] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:48.864708 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:48.866282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:48.881072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f605-c7ae-4f97-925b-0924b4a89b62 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:48.885026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:48.885673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:48.885967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:48.886249 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:48.888614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:48.899394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:48.901540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:48.910856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1261/2526] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:51.362784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:51.376940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd485357-a164-42e0-a6d2-b099bfdc97e3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:51.381134 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:51.382039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:51.382319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:51.382613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:51.385002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:51.400224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:51.401686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:51.412568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1266/2527] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:02:54.895084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:54.896743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:54.913063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f21d11-22f5-4193-861c-36f8d11c612b service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:54.918762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:54.919646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:54.919969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:54.920246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] PATH_INFO: `/v3/users/6ead76e4f3564d6eb614bb911ccaf674` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:54.922807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ead76e4f3564d6eb614bb911ccaf674)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:54.924818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:54.985014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ead76e4f3564d6eb614bb911ccaf674'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:54.996900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ead76e4f3564d6eb614bb911ccaf674'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:54.997681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ead76e4f3564d6eb614bb911ccaf674'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.004382 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ead76e4f3564d6eb614bb911ccaf674'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.013450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9da7bae3-c374-487d-8d12-d79b5d8e092e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ead76e4f3564d6eb614bb911ccaf674'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.019824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1262/2528] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:02:54 2022] DELETE /identity/v3/users/6ead76e4f3564d6eb614bb911ccaf674 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:55.208674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.224809 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99a425-4188-4e36-9d79-cdfa2c1763fd service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.229274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:55.301186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.302072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.302351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.302635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] PATH_INFO: `/v3/projects/a3160540122441ce970560d7a3db2521` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.303801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3160540122441ce970560d7a3db2521)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.305793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.354763 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3160540122441ce970560d7a3db2521', 'request_id': 'req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.360164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3160540122441ce970560d7a3db2521', 'request_id': 'req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.366323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3160540122441ce970560d7a3db2521', 'request_id': 'req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.380779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] The token cache is being invalidate because project a3160540122441ce970560d7a3db2521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:02:55.381074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.381733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-9d0ec797-0fac-4d49-9a55-56d48d74e4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:02:55.383272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1267/2529] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:02:55 2022] DELETE /identity/v3/projects/a3160540122441ce970560d7a3db2521 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:02:55.388528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.388883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.389220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.407248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:55.408484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:55.470839 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1263/2530] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:55.479620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035303be-486d-42c1-9322-693d41aa9208 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.480084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035303be-486d-42c1-9322-693d41aa9208 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.480376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035303be-486d-42c1-9322-693d41aa9208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.500518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-035303be-486d-42c1-9322-693d41aa9208 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:55.501667 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-035303be-486d-42c1-9322-693d41aa9208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:55.511911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1268/2531] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:02:55.516409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.526562 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b59a0-76ba-477b-8adf-73423754bce7 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.531985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.532642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.532984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.533253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.533970 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:02:55.534274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.535909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.548510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1264/2532] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:02:55.552923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035303be-486d-42c1-9322-693d41aa9208 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.562615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035303be-486d-42c1-9322-693d41aa9208 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.568414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.569401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.569684 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.569975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.570675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.572712 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.608003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1269/2533] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:02:55 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:02:55.609573 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.619804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfdd64-b827-4e89-91b2-6a1b1f2747c4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.625141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:55.680793 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.681665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.681952 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.682232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.682968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.684911 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.705620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1265/2534] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:02:55 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:02:55.709984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.721228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5319-e1db-4d64-8405-7e9e3464ebe0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.726118 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.726799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.727083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.727360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.728348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.729877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.741084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1270/2535] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:55.746729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.757973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd56c5-b14b-4bdc-b291-2ea448aaa9f0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.763204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.763937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.764223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.768063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.768063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.768063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223ce29-1e99-467c-9833-dbccfc7843bd admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.779150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1266/2536] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:02:55.784178 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.794690 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff22df3-65ab-4c02-ace0-b791da59743d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.799387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.800324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.800609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.800888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] PATH_INFO: `/v3/projects/1bf3e1149d124a62b0984ec2598db78e/users/0141317702184143b85a9dce65ca3b84/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.817769 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf3e1149d124a62b0984ec2598db78e, user_id=0141317702184143b85a9dce65ca3b84, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:55.820700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:55.835093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1271/2537] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:02:55 2022] PUT /identity/v3/projects/1bf3e1149d124a62b0984ec2598db78e/users/0141317702184143b85a9dce65ca3b84/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:02:55.840457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dea74-6312-4b14-aa02-f4deacda97f5 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.840746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dea74-6312-4b14-aa02-f4deacda97f5 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:55.841020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dea74-6312-4b14-aa02-f4deacda97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:55.860511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-760dea74-6312-4b14-aa02-f4deacda97f5 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:02:55.861722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-760dea74-6312-4b14-aa02-f4deacda97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:55.922129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1267/2538] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:02:55.932915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:02:55.934822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:02:55.971216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:02:55.994565 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775c34-802c-4758-87f1-cb9e79cf18a7 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:02:55.998845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:02:55.999503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:02:55.999935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:02:56.000217 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:02:56.010429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:02:56.011816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:02:56.022030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1272/2539] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:03:00.341093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bae84-da64-4156-943d-dfac8e32a47b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:00.341597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bae84-da64-4156-943d-dfac8e32a47b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:00.341714 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bae84-da64-4156-943d-dfac8e32a47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:00.369376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-d47bae84-da64-4156-943d-dfac8e32a47b None None] MFA Rules not processed for user `0141317702184143b85a9dce65ca3b84`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:00.370596 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d47bae84-da64-4156-943d-dfac8e32a47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:00.439211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1268/2540] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:00.455622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:00.457319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:00.537946 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c8496-6a0c-4079-a325-efc89eea0a5b service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:00.543824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:00.544511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:00.544793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:00.545071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:00.554799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:00.556523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:00.566286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1273/2541] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:01.979184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:01.979718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:01.979843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:02.001127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] MFA Rules not processed for user `0141317702184143b85a9dce65ca3b84`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:02.002425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:02.010604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1269/2542] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:02.400845 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:02.404218 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:02.432293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1873fae-8f05-4d67-b3ae-86d857c79d39 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:02.437621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:02.438988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:02.439215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:02.439489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:02.442059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:02.454331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:02.456384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:02.475170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1274/2543] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:11.536134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:11.546786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe594ad-4f63-432d-9582-f9968a370ab4 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:11.550752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:11.551727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:11.552235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:11.552683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:11.554884 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:11.586800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:11.588409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:11.593460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:11.605999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1270/2544] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:12.470332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=51 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:14.917871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:14.920243 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:14.991639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:14.993404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:15.018930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7d339-93bc-4933-b0d9-500e350a25bb service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.035395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.049149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.050716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.051126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] PATH_INFO: `/v3/users/3e4012911dad42edbffead00e0fd55e6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.055292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4012911dad42edbffead00e0fd55e6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.060948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.083751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1bc15-b56b-4b37-a887-d5c3832b01db service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.089323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.090003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.090284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.090557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.092714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:15.106327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.108153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.119822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1275/2545] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:15.155269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4012911dad42edbffead00e0fd55e6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.168520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4012911dad42edbffead00e0fd55e6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.169268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4012911dad42edbffead00e0fd55e6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.176639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4012911dad42edbffead00e0fd55e6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.185730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-ce8b90e4-80b1-44ab-bdf3-873c8e3c5f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4012911dad42edbffead00e0fd55e6'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.192711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1271/2546] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:14 2022] DELETE /identity/v3/users/3e4012911dad42edbffead00e0fd55e6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:03:15.374623 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.386091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603e33a-4347-4f37-905b-a9b6d6f6bc40 service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.453390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.454269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.454561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.454830 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] PATH_INFO: `/v3/projects/2ff1f327753b4708b0940964aa3d761c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.456067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff1f327753b4708b0940964aa3d761c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.458074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.507857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff1f327753b4708b0940964aa3d761c', 'request_id': 'req-ef2bc9a3-0525-4625-b55b-ae9b5425042d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.513751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff1f327753b4708b0940964aa3d761c', 'request_id': 'req-ef2bc9a3-0525-4625-b55b-ae9b5425042d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.519756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff1f327753b4708b0940964aa3d761c', 'request_id': 'req-ef2bc9a3-0525-4625-b55b-ae9b5425042d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.533225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] The token cache is being invalidate because project 2ff1f327753b4708b0940964aa3d761c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:15.533510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.534249 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ef2bc9a3-0525-4625-b55b-ae9b5425042d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:15.535924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1276/2547] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:15 2022] DELETE /identity/v3/projects/2ff1f327753b4708b0940964aa3d761c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:15.541409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.541829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.542122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.560520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:15.561725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:15.624913 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1272/2548] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:03:15.633193 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe331-d872-403b-9c2e-57431621a80e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.633483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe331-d872-403b-9c2e-57431621a80e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.633761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befe331-d872-403b-9c2e-57431621a80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.654085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-0befe331-d872-403b-9c2e-57431621a80e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:15.655227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0befe331-d872-403b-9c2e-57431621a80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:15.665090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1277/2549] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:15.670175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.684416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d4d05-825d-472b-a765-3bf3dd5c59b8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.690320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.690980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.691262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.691536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.692313 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:15.692621 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.694262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.706254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1273/2550] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:03:15.710627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe331-d872-403b-9c2e-57431621a80e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.720978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befe331-d872-403b-9c2e-57431621a80e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.726073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.726725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.727006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.727277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.727995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.729906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.740115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:15.759395 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1278/2551] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:15 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:15.764261 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.774539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451d68-7c63-4dca-be90-f2e98990ee1a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.826707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:15.833697 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.834425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.834713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.834998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.836004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.839278 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.866046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1274/2552] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:15 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:15.867282 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.878669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f8ba-3dc4-450a-9215-400b08769672 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.883827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.884542 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.884859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.885164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.886007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.887637 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.898800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1279/2553] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:15.903317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.914614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434edb-4f55-4386-968d-6be5c3fc0495 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.919895 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.920899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.921186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.921460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.922191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.923835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad922ed-4cfb-4d06-b8cd-0f5890c4bbb7 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.936610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1275/2554] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:15.940744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:15.951222 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700226d-c941-41bf-a279-1b4602a2588e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:15.956176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:15.957059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:15.957331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:15.957597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] PATH_INFO: `/v3/projects/8a573a1e5a984820a799249021a88b65/users/eb1827e6ddd5441684348ded3790735f/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:15.973750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a573a1e5a984820a799249021a88b65, user_id=eb1827e6ddd5441684348ded3790735f, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:15.976453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:15.990851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1280/2555] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:15 2022] PUT /identity/v3/projects/8a573a1e5a984820a799249021a88b65/users/eb1827e6ddd5441684348ded3790735f/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:16.003409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:16.003409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:16.003409 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:16.032214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] MFA Rules not processed for user `eb1827e6ddd5441684348ded3790735f`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:16.033456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:16.100850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1276/2556] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:16.112467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:16.114526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:16.177029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc041cd-5417-4637-842e-a4d07985c213 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:16.180872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:16.181537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:16.181824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:16.182279 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:16.191488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:16.193131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:16.202940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1281/2557] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:17.708861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:17.710492 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:17.802580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d830802-0edb-4192-8c58-4b609dbd2452 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:17.808575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:17.809226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:17.809509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:17.809774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:17.811772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:17.825587 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:17.827472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05d83-c779-4f03-a47d-f3e53a21709a service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:17.839403 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1277/2558] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:24.701077 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:24.772552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aa5dc-dc64-48bc-bf0e-2c5faf0f3b39 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:24.780728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:24.781992 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:24.782293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:24.782552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] PATH_INFO: `/v3/users/eb1827e6ddd5441684348ded3790735f` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:24.783867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1827e6ddd5441684348ded3790735f)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:24.785881 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:24.844981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1827e6ddd5441684348ded3790735f'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.854846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:24.857253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1827e6ddd5441684348ded3790735f'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.858306 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1827e6ddd5441684348ded3790735f'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.867384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1827e6ddd5441684348ded3790735f'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.877605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1827e6ddd5441684348ded3790735f'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:24.883634 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1282/2559] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:24 2022] DELETE /identity/v3/users/eb1827e6ddd5441684348ded3790735f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:24.888877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:24.889764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:24.889764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:24.907442 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:24.908661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:24.955627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:24.971182 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1278/2560] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:03:24.981005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:24.982538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:25.048209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c3b7e-eb4a-4bf3-8ba0-4b7ce76418df admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.052880 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.053526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.053805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.054059 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.064504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.065935 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc38d2-7e71-4a18-aa1e-550f40eeb24e service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.076423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1283/2561] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:25.360746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.362183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:25.376314 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad54913-de75-474c-9e14-ff209d8fcdbc None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.382149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.383045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.383324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.383601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] PATH_INFO: `/v3/projects/8a573a1e5a984820a799249021a88b65` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.384899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a573a1e5a984820a799249021a88b65)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.387282 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.437178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a573a1e5a984820a799249021a88b65', 'request_id': 'req-8050724c-57bf-4304-8dcc-8c74a383a9cb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:25.443627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a573a1e5a984820a799249021a88b65', 'request_id': 'req-8050724c-57bf-4304-8dcc-8c74a383a9cb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:25.450243 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a573a1e5a984820a799249021a88b65', 'request_id': 'req-8050724c-57bf-4304-8dcc-8c74a383a9cb'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:25.469631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] The token cache is being invalidate because project 8a573a1e5a984820a799249021a88b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:25.469921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:25.470558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8050724c-57bf-4304-8dcc-8c74a383a9cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:25.472168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1279/2562] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:25 2022] DELETE /identity/v3/projects/8a573a1e5a984820a799249021a88b65 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:25.477186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.477474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.477753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.497784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:25.499145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:25.562635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1284/2563] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:25.572173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.572485 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.572824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.592445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:25.593631 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:25.602522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1280/2564] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:25.606897 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.616693 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78edd1-523a-40e9-ac16-97ff0aefc9e8 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.621398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.622072 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.622353 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.623172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.623608 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:25.623917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.625581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.637139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1285/2565] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 11:03:25.641542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.651629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e0429-f744-47c3-81a4-d8f4803e8da7 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.658104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.658594 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.658870 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.659166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.659966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.661914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.693215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1281/2566] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:25 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:25.697548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.707755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d661-d014-4d4f-85e9-81aec3c6ab21 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.766198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.766865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.767157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.767432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.768150 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.770085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.787415 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:25.793445 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1286/2567] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:25 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:25.798513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.809420 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ab6b-b2bd-48e8-ab80-f091fc5a7735 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.815768 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.816477 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.816786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.817065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.817795 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.819418 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.830115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1282/2568] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:25.834209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.844853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f3cc5-a8cf-4c0e-a06b-659123d2664f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.850553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.851275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.851672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.852001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.852732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.854332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76e1e2-fb8f-4ec4-9616-d15acaccce44 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.865109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1287/2569] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:03:25.869437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:25.880436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ecdeb7-ade7-4cdd-b265-019c51702bac admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:25.886318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:25.887324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.887513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.887819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] PATH_INFO: `/v3/projects/15d6541b8c1440068f5123254c7843dc/users/cc7273463a0c4a1baa19d4fd3de42fd8/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.904220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d6541b8c1440068f5123254c7843dc, user_id=cc7273463a0c4a1baa19d4fd3de42fd8, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:25.907357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:25.925643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1283/2570] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:25 2022] PUT /identity/v3/projects/15d6541b8c1440068f5123254c7843dc/users/cc7273463a0c4a1baa19d4fd3de42fd8/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:25.936816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:25.937416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:25.937924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:25.966954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] MFA Rules not processed for user `cc7273463a0c4a1baa19d4fd3de42fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:25.968253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:26.044270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1288/2571] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:26.049620 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.120376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22ea6-c748-4fdc-9161-1d1e8e11be36 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.127276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.127963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.128237 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.128506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.129201 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.131123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.146956 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:26.164622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1284/2572] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:26 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:03:26.170509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.182090 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87a7a2-da67-400d-ace0-a92f2f288b98 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.240595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.241255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.241535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.241799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.242513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.244457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.265958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1289/2573] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:26 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:26.272433 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.284218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd028c0-cce5-47db-b130-9d43e5495bac admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.289360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.290018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.290295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.290564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.291289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.292940 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.303595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1285/2574] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:26.307991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.318555 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b92607-ef49-43fc-8e74-e350678cae0c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.324305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.325226 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.325512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.325860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] PATH_INFO: `/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6/users/ef5763f96abb4a0a88820164b5d4cfa2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.341888 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd02f68e24d4716a0dcb8bcb838c1d6, user_id=ef5763f96abb4a0a88820164b5d4cfa2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.344454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.359519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1290/2575] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:26 2022] PUT /identity/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6/users/ef5763f96abb4a0a88820164b5d4cfa2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:26.363607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.373707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8e7d-e7ad-4823-87cc-94147aeaba7b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.431100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.431684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.432007 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.432279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.432974 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.434580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd26abe-cf03-4a13-94ae-4cd51be166ab admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.444984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1286/2576] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:26.449384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.460467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed788d6-db6c-4559-bafa-5761ed303197 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.465801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.466768 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.467061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.467337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] PATH_INFO: `/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6/users/ef5763f96abb4a0a88820164b5d4cfa2/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.469471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd02f68e24d4716a0dcb8bcb838c1d6, user_id=ef5763f96abb4a0a88820164b5d4cfa2, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.472184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.483322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1291/2577] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:26 2022] PUT /identity/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6/users/ef5763f96abb4a0a88820164b5d4cfa2/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:26.493563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.493850 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.494138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.522169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] MFA Rules not processed for user `ef5763f96abb4a0a88820164b5d4cfa2`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:26.523378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:26.599629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1287/2578] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:26.611975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.613899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:26.646384 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:26.675170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccc961-8fe6-4339-a522-c35164e2eff3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.679898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.680553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.680837 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.681165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.691421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.693009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.703015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1292/2579] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:26.786067 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:26.797630 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceb4bd-6089-463b-82d1-b88b53f19817 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:26.801804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:26.802450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:26.802731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:26.803012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:26.863665 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:26.865194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d7f28-7209-4a56-98f1-7cc0356e508a service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:26.874717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1288/2580] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:36.698588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=13 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:36.870435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:40.443747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:40.506356 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4aa65-d7b1-435f-9bab-ef90e85d02c9 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:40.511905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:40.512788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:40.513066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:40.513337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] PATH_INFO: `/v3/users/cc7273463a0c4a1baa19d4fd3de42fd8` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:40.514659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7273463a0c4a1baa19d4fd3de42fd8)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:40.516656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:40.567128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7273463a0c4a1baa19d4fd3de42fd8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:40.579850 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7273463a0c4a1baa19d4fd3de42fd8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:40.580588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7273463a0c4a1baa19d4fd3de42fd8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:40.588765 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7273463a0c4a1baa19d4fd3de42fd8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:40.597950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7273463a0c4a1baa19d4fd3de42fd8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:40.604366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1293/2581] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:40 2022] DELETE /identity/v3/users/cc7273463a0c4a1baa19d4fd3de42fd8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:03:40.608967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:40.609460 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:40.609557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:40.627367 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:40.628613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:40.694930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1289/2582] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:40.708394 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:40.710167 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:40.770376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fa382-e7f3-4700-8a3a-bf198952a072 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:40.775162 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:40.775848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:40.776212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:40.776593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:40.786467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:40.788017 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:40.797363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1294/2583] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:41.075785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:41.077467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:41.093878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed5043-062c-4e59-a5f6-7770441e712d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:41.100291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:41.101117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.101242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.101508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] PATH_INFO: `/v3/projects/15d6541b8c1440068f5123254c7843dc` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.102808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d6541b8c1440068f5123254c7843dc)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:41.104816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:41.156702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d6541b8c1440068f5123254c7843dc', 'request_id': 'req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.163705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d6541b8c1440068f5123254c7843dc', 'request_id': 'req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.169364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d6541b8c1440068f5123254c7843dc', 'request_id': 'req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.183042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] The token cache is being invalidate because project 15d6541b8c1440068f5123254c7843dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:41.183333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.183975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.185522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1290/2584] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:41 2022] DELETE /identity/v3/projects/15d6541b8c1440068f5123254c7843dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:41.189642 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:41.199246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c53aa-5014-424a-b075-8f7dad3231cf service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:41.217566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:41.259329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:41.260251 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.260538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.260815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] PATH_INFO: `/v3/users/ef5763f96abb4a0a88820164b5d4cfa2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.262014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5763f96abb4a0a88820164b5d4cfa2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:41.264051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:41.322238 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5763f96abb4a0a88820164b5d4cfa2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.336407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5763f96abb4a0a88820164b5d4cfa2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.337501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5763f96abb4a0a88820164b5d4cfa2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.345322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5763f96abb4a0a88820164b5d4cfa2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.356058 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f1dd5093-4e9d-4c02-92af-d7d8229b5fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5763f96abb4a0a88820164b5d4cfa2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.362413 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1295/2585] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:41 2022] DELETE /identity/v3/users/ef5763f96abb4a0a88820164b5d4cfa2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:41.641229 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:41.642878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:41.656728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29dd07-52ac-458d-bd37-45c4e3e1a08e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:41.716312 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:41.717155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.717430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.717719 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] PATH_INFO: `/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.718915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd02f68e24d4716a0dcb8bcb838c1d6)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:41.720966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:41.755622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:41.774373 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd02f68e24d4716a0dcb8bcb838c1d6', 'request_id': 'req-4300a733-423f-43bb-a01e-beec2c357b82'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.781089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd02f68e24d4716a0dcb8bcb838c1d6', 'request_id': 'req-4300a733-423f-43bb-a01e-beec2c357b82'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.787027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd02f68e24d4716a0dcb8bcb838c1d6', 'request_id': 'req-4300a733-423f-43bb-a01e-beec2c357b82'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.803763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] The token cache is being invalidate because project 7fd02f68e24d4716a0dcb8bcb838c1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:41.804270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.805011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4300a733-423f-43bb-a01e-beec2c357b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:41.806701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1291/2586] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:41 2022] DELETE /identity/v3/projects/7fd02f68e24d4716a0dcb8bcb838c1d6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:41.811846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.812365 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.812596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.831057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:41.832323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:41.892248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1296/2587] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:41.900389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.900684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.900969 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.920183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:41.921397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:41.930018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1292/2588] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:03:41.934907 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:41.945420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff635f0c-6e66-4f4c-a40d-afde486ce457 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:41.951023 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:41.952294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.952576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.952853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.953591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:41.953899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:41.955554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:41.971898 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1297/2589] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:03:41.973144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:41.982837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c1185-ddb6-468f-a58e-6141538dd6e3 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:41.987709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:41.988502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:41.988783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:41.989057 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:41.989945 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:41.991998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.021401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1293/2590] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:41 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:42.026061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.039589 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6d239-6b27-4d5b-b0b1-85126a15bcf5 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.068681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:42.103426 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.104159 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.104447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.104722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.105420 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.107357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.129725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1298/2591] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:42 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:03:42.134173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.144819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce742d6-1225-4adb-81c0-d34c0fea246b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.150276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.150949 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.151236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.151516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.152288 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.153900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.165873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1294/2592] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:42.170127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.180737 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358003e3-20d9-43ae-a13f-b30ef5eb1675 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.186404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.187066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.187349 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.187628 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.188398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.190000 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61740f7-8471-4364-b47b-463717ffd3b5 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.200868 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1299/2593] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:42.205186 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.215755 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136aca3-d53e-4acd-a74e-f59546190c22 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.221155 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.222123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.222340 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.222613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] PATH_INFO: `/v3/projects/d7bc20748012486ca4061ac9091471c6/users/b5bbc1ae566240a19fbb555173cac65a/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.237683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bc20748012486ca4061ac9091471c6, user_id=b5bbc1ae566240a19fbb555173cac65a, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.240938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.257002 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1295/2594] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:42 2022] PUT /identity/v3/projects/d7bc20748012486ca4061ac9091471c6/users/b5bbc1ae566240a19fbb555173cac65a/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:42.267986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.268377 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.268779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.299858 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] MFA Rules not processed for user `b5bbc1ae566240a19fbb555173cac65a`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:42.301103 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:42.372360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1300/2595] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:42.376929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.444347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82b219-3edd-4c19-b443-bdcbd77ce637 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.449188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.449847 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.450113 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.450407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.451128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.453056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.482737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1296/2596] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:42 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:42.488717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.501330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa1b99-8abc-422d-abf1-a820f47d315d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.568923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.569640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.569917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.570596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.570877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.572835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.596075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1301/2597] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:42 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:42.597422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.608543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895aa52-6689-4bed-bec6-8f361c11fa73 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.613152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.613809 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.614089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.614474 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.615400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.617096 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.628827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1297/2598] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:42.633127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.643927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57f5ea-4bf0-4c88-a0cd-72a45e7bacca admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.649483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.650375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.650660 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.650925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] PATH_INFO: `/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172/users/cc98094aa0b74369b82afde516dae918/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.667296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b82d3b8ba3b4a0ea7ec2c61c2dd9172, user_id=cc98094aa0b74369b82afde516dae918, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.670221 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.685191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1302/2599] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:42 2022] PUT /identity/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172/users/cc98094aa0b74369b82afde516dae918/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:42.689200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.700864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69d248-d473-4dbf-960b-215bafb37a4f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.721486 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:42.762861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.763446 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.763729 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.764037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.764762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.766390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84de832-4e0d-474e-825d-ea035b0ca412 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.778211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1298/2600] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:03:42.782861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.788651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:42.794626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52fcc3-5f5e-4684-9e34-2fd947051e73 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:42.799852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:42.800766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.801047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.801317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] PATH_INFO: `/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172/users/cc98094aa0b74369b82afde516dae918/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.803503 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b82d3b8ba3b4a0ea7ec2c61c2dd9172, user_id=cc98094aa0b74369b82afde516dae918, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:42.806201 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:42.816853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1303/2601] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:42 2022] PUT /identity/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172/users/cc98094aa0b74369b82afde516dae918/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:42.828292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:42.828610 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:42.828888 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:42.857427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] MFA Rules not processed for user `cc98094aa0b74369b82afde516dae918`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:42.858655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:42.943054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1299/2602] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:42.954113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:42.956003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:43.015829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e50c8-d43e-4cfb-9dcc-5fbff33aeda2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:43.020105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:43.020903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:43.020903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:43.021463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:43.030829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:43.032627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf74533-6d6d-4f85-ba06-7b7f6c11868a service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:43.042121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1304/2603] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:44.460654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:44.524022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57baa53-6325-4a8f-b031-922d589630ec None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:44.528559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:44.529439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:44.529720 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:44.529994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] PATH_INFO: `/v3/users/b5bbc1ae566240a19fbb555173cac65a` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:44.531181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5bbc1ae566240a19fbb555173cac65a)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:44.533501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:44.613449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5bbc1ae566240a19fbb555173cac65a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:44.615185 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:44.628887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5bbc1ae566240a19fbb555173cac65a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:44.629639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5bbc1ae566240a19fbb555173cac65a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:44.636576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5bbc1ae566240a19fbb555173cac65a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:44.644860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5bbc1ae566240a19fbb555173cac65a'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:44.650943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1300/2604] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:44 2022] DELETE /identity/v3/users/b5bbc1ae566240a19fbb555173cac65a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:44.655670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:44.656246 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:44.656334 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:44.675489 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:44.676951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:44.743944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1305/2605] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:03:44.754756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:44.756359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:44.827077 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530ba25-8020-437d-9871-d1f0533c2b7a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:44.830824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:44.831465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:44.833175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:44.833478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:44.844298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:44.846140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:44.856359 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1301/2606] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:45.154312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:45.155985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:45.177051 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e468fc6-85fa-4071-b556-d55f677b86f6 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:45.183098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:45.184179 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:45.184443 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:45.184686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] PATH_INFO: `/v3/projects/d7bc20748012486ca4061ac9091471c6` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:45.185942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bc20748012486ca4061ac9091471c6)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:45.187906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:45.239230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bc20748012486ca4061ac9091471c6', 'request_id': 'req-fe96d40c-68fa-467e-9245-dabd4e29c2a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.246173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bc20748012486ca4061ac9091471c6', 'request_id': 'req-fe96d40c-68fa-467e-9245-dabd4e29c2a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.252709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bc20748012486ca4061ac9091471c6', 'request_id': 'req-fe96d40c-68fa-467e-9245-dabd4e29c2a8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.266797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] The token cache is being invalidate because project d7bc20748012486ca4061ac9091471c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:45.267087 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.267685 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.269410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1306/2607] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:45 2022] DELETE /identity/v3/projects/d7bc20748012486ca4061ac9091471c6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:45.274072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:45.285635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c5aa-1470-45ee-a87c-856975e150c0 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:45.347591 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:45.348495 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:45.348805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:45.349075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] PATH_INFO: `/v3/users/cc98094aa0b74369b82afde516dae918` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:45.350413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc98094aa0b74369b82afde516dae918)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:45.352414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:45.402997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc98094aa0b74369b82afde516dae918'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.414725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc98094aa0b74369b82afde516dae918'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.415390 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc98094aa0b74369b82afde516dae918'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.420274 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:45.423689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc98094aa0b74369b82afde516dae918'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.433195 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-a6d6e4b8-a823-4d90-9567-8b9602d6cff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc98094aa0b74369b82afde516dae918'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.439889 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1302/2608] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:45 2022] DELETE /identity/v3/users/cc98094aa0b74369b82afde516dae918 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:45.738960 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:45.740913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:45.757335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96d40c-68fa-467e-9245-dabd4e29c2a8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:45.772122 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:45.822139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:45.823074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:45.823348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:45.823616 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] PATH_INFO: `/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:45.824959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b82d3b8ba3b4a0ea7ec2c61c2dd9172)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:45.827037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:45.876451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b82d3b8ba3b4a0ea7ec2c61c2dd9172', 'request_id': 'req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.882404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b82d3b8ba3b4a0ea7ec2c61c2dd9172', 'request_id': 'req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.888698 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b82d3b8ba3b4a0ea7ec2c61c2dd9172', 'request_id': 'req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.903049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] The token cache is being invalidate because project 3b82d3b8ba3b4a0ea7ec2c61c2dd9172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:45.903331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.903988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e9b0c5c5-ae01-49be-861e-a0bdaed80d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:45.905550 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1307/2609] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:45 2022] DELETE /identity/v3/projects/3b82d3b8ba3b4a0ea7ec2c61c2dd9172 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:45.910766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:45.911245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:45.911432 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:45.930760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:45.931859 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:45.998862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1303/2610] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:46.010625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.011037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.011400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.034731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:46.036004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:46.044988 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1308/2611] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:46.049476 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.060391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f109e-8bcb-4f2b-9b08-6fa7b9eb7cb8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.067760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.068520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.068801 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.069075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.069805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:46.070142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.071752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.084175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1304/2612] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:03:46.089133 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.099829 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5208c-5273-4ba2-8df7-71cc2262d0db None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.105434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.106172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.106450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.106720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.107467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.109652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.139918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1309/2613] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:46.143519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.154722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7096724-92ed-48f0-b1c8-84e8308c62fb admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.215721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.215964 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.216223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.216501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.217239 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.219181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.244128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1305/2614] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:46 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:03:46.248300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.258902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2697adc-8dd4-4f48-a469-296c79ae3c8a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.263984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.264666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.264950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.265227 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.266010 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.267595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.278357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1310/2615] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:46.282745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.292955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73ef7d-9cf6-48c3-bf7f-188be51cd96d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.297816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.298468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.298756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.299031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.299762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.301425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31f2ea-e972-40bf-8070-a6aea6fd159e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.313320 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1306/2616] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:46.317665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.327984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640c08-92c0-4c59-812e-cb93ce53d0ff admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.332889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.333853 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.334132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.334407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] PATH_INFO: `/v3/projects/4be8f1fd100540e185a2110bdeb59962/users/5c3ce13b61074c329fec2741fc3d5d75/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.348751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be8f1fd100540e185a2110bdeb59962, user_id=5c3ce13b61074c329fec2741fc3d5d75, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.351573 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.366757 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1311/2617] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:46 2022] PUT /identity/v3/projects/4be8f1fd100540e185a2110bdeb59962/users/5c3ce13b61074c329fec2741fc3d5d75/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:46.379236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715dec-0f3a-4b24-bc1d-e2ee5088c689 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.379615 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715dec-0f3a-4b24-bc1d-e2ee5088c689 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.379976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b715dec-0f3a-4b24-bc1d-e2ee5088c689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.405976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8b715dec-0f3a-4b24-bc1d-e2ee5088c689 None None] MFA Rules not processed for user `5c3ce13b61074c329fec2741fc3d5d75`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:46.407549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8b715dec-0f3a-4b24-bc1d-e2ee5088c689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:46.454427 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:46.479639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1307/2618] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:46.491075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:46.492895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:46.541931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:46.555281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f7cb-6e0f-489c-b17e-4f2d4d84a134 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:46.559212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:46.559852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:46.560148 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:46.560540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:46.569053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:46.570512 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:46.579506 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1312/2619] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:56.476734 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=8 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:56.575959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=7 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:56.955527 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bea10-e9c8-47aa-aa32-185b2854673b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:56.956014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bea10-e9c8-47aa-aa32-185b2854673b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:56.956435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bea10-e9c8-47aa-aa32-185b2854673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:56.978440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-554bea10-e9c8-47aa-aa32-185b2854673b None None] MFA Rules not processed for user `75a9903b3597458f8f5af4d06cd07d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:56.979646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-554bea10-e9c8-47aa-aa32-185b2854673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:57.042356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1308/2620] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:57.245559 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:57.313209 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7559b6-8323-45e8-b752-2fa4c0e92998 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:57.318100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:57.319037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:57.319323 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:57.319604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] PATH_INFO: `/v3/users/5c3ce13b61074c329fec2741fc3d5d75` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:57.320873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3ce13b61074c329fec2741fc3d5d75)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:57.322841 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:57.374026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3ce13b61074c329fec2741fc3d5d75'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.386194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3ce13b61074c329fec2741fc3d5d75'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.386953 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3ce13b61074c329fec2741fc3d5d75'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.394948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3ce13b61074c329fec2741fc3d5d75'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.403501 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3ce13b61074c329fec2741fc3d5d75'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.410258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1313/2621] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:03:57 2022] DELETE /identity/v3/users/5c3ce13b61074c329fec2741fc3d5d75 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:57.415084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:57.415455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:57.415742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:57.435183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:57.436385 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:57.498133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1309/2622] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:57.508328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:57.509755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:57.569386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf1e1-3cd6-45a2-948c-1ef3fd6a0ffa admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:57.573324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:57.573981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:57.574410 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:57.574762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:57.586092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:57.587171 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a549-d1fc-4bc8-b352-3116a64d70ed service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:57.599191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1314/2623] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:03:57.906384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:57.908284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:57.923120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d09660-ec01-4e1e-85ef-08a729344dc7 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:57.930018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:57.931254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:57.931901 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:57.932388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] PATH_INFO: `/v3/projects/4be8f1fd100540e185a2110bdeb59962` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:57.933554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be8f1fd100540e185a2110bdeb59962)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:57.935907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:57.987758 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be8f1fd100540e185a2110bdeb59962', 'request_id': 'req-2576599e-e002-4fe7-8e8d-e42b93b43bc8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.993998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be8f1fd100540e185a2110bdeb59962', 'request_id': 'req-2576599e-e002-4fe7-8e8d-e42b93b43bc8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:57.999995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be8f1fd100540e185a2110bdeb59962', 'request_id': 'req-2576599e-e002-4fe7-8e8d-e42b93b43bc8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:58.015672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] The token cache is being invalidate because project 4be8f1fd100540e185a2110bdeb59962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:03:58.015995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:58.016628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2576599e-e002-4fe7-8e8d-e42b93b43bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:03:58.018219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1310/2624] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:03:57 2022] DELETE /identity/v3/projects/4be8f1fd100540e185a2110bdeb59962 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:58.023062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.023350 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.023626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.043831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:58.045105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:58.064173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:58.111751 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1315/2625] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:58.120619 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.120914 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.121193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.140688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:58.141928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:58.150110 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1311/2626] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:58.154681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.164987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527be6-89f8-4fe3-b0e9-9eefeb63c8e2 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.170062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.170726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.171008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.171315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.172084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:03:58.172396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.174026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.189905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1316/2627] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:03:58.191202 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.200856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d28de6-68ef-435f-94b4-82eb2e3745d6 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.205466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.206123 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.206406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.206678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.207391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.209323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.232036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:58.239368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1312/2628] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:58.243487 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.254449 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70561-7a1e-4ae8-81bc-a365c0312be4 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.311622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.312319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.312597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.312844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.313558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.315486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.338280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1317/2629] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:58.342652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.353296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713598a-d56b-4b54-bd61-80cc7c5c487c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.358968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.359628 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.359918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.360291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.360968 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.362608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.373393 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1313/2630] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:58.378083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.393633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d14b-79f1-40fc-98d2-e0848ebea19b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.401536 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.402470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.402836 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.403241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.404046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.405675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b9f56-94ac-4e82-8c0f-2fc58ed5e0dc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.416918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1318/2631] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:58.421639 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.432011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736272-3de5-4a1f-a5ce-eb540421ac38 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.437116 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.438001 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.438285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.438563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] PATH_INFO: `/v3/projects/c3f374b08ec944d19a382bef07d0345a/users/a1127b6194284720ae065e88c6cfbaab/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.454097 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f374b08ec944d19a382bef07d0345a, user_id=a1127b6194284720ae065e88c6cfbaab, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.456851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.471405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1314/2632] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:58 2022] PUT /identity/v3/projects/c3f374b08ec944d19a382bef07d0345a/users/a1127b6194284720ae065e88c6cfbaab/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:58.481985 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.482274 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.482569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.510910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] MFA Rules not processed for user `a1127b6194284720ae065e88c6cfbaab`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:58.512074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:58.594813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1319/2633] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:58.599292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.670524 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3660fcb-0fdf-4136-94af-8878d9e74358 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.675987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.676816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.676960 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.677139 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.677919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.679823 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.718783 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1315/2634] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:58.722997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.736451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b1df1-4099-4b44-9967-89613b3fb405 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.750786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:58.805030 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.806052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.806332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.806613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.807327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.809441 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.831107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1320/2635] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:03:58 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:03:58.835254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.846022 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c5995-bfa4-4312-8444-8b4b7824d68f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.850607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.851272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.851555 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.851830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.852592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.854213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.865667 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1316/2636] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:58.869970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.880822 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc55e2-edef-42db-970b-6a76b9eebec6 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:58.885786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:58.886673 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:58.886949 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:58.887220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] PATH_INFO: `/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9/users/15da0edcaa9e4e098185b3345d2c9161/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:58.903327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ca38bf26cf48928bad41a3f90ca2b9, user_id=15da0edcaa9e4e098185b3345d2c9161, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:58.906052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:58.921759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1321/2637] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:58 2022] PUT /identity/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9/users/15da0edcaa9e4e098185b3345d2c9161/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:58.926548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:58.938883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f1bef-2088-43e6-8c7f-1270e9678222 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:59.000038 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:59.000715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:59.000996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:59.001294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:59.002085 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:59.003718 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aadc37-fbdd-47f6-a6d6-eeaacd634b50 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:59.014873 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1317/2638] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:03:59.019225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:59.030480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6239d-9b61-45df-9ae4-1d470ac60d32 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:59.035864 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:59.036795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:59.037082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:59.037331 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] PATH_INFO: `/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9/users/15da0edcaa9e4e098185b3345d2c9161/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:59.039439 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ca38bf26cf48928bad41a3f90ca2b9, user_id=15da0edcaa9e4e098185b3345d2c9161, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:59.042128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:59.053980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1322/2639] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:03:59 2022] PUT /identity/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9/users/15da0edcaa9e4e098185b3345d2c9161/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:03:59.067166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25843-6459-491f-afc7-832facdada6e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:59.067461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25843-6459-491f-afc7-832facdada6e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:59.067746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff25843-6459-491f-afc7-832facdada6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:59.095247 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-bff25843-6459-491f-afc7-832facdada6e None None] MFA Rules not processed for user `15da0edcaa9e4e098185b3345d2c9161`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:03:59.096575 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-bff25843-6459-491f-afc7-832facdada6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:59.120985 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:03:59.168991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1318/2640] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:03:59.180651 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:03:59.182355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:03:59.242033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457b45d-7313-4219-884f-c319312eaed2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:03:59.245718 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:03:59.246689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:03:59.247230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:03:59.247560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:03:59.309566 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:03:59.311029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bc95a-eacd-452f-98b6-0f315e9e826c service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:03:59.320195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1323/2641] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:03:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:01.897907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff25843-6459-491f-afc7-832facdada6e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:01.971838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff25843-6459-491f-afc7-832facdada6e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:01.976484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:01.977419 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:01.977703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:01.977986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] PATH_INFO: `/v3/users/a1127b6194284720ae065e88c6cfbaab` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:01.979404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1127b6194284720ae065e88c6cfbaab)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:01.981586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:02.029360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1127b6194284720ae065e88c6cfbaab'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.041478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1127b6194284720ae065e88c6cfbaab'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.042066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1127b6194284720ae065e88c6cfbaab'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.048804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1127b6194284720ae065e88c6cfbaab'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.057991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1127b6194284720ae065e88c6cfbaab'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.064307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1319/2642] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:01 2022] DELETE /identity/v3/users/a1127b6194284720ae065e88c6cfbaab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:02.068870 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:02.069409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:02.069409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:02.086903 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:02.088307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:02.109029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:02.160357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1324/2643] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:04:02.170253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:02.171897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:02.219276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:02.236083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830e542-b7c4-497e-802f-ed34252a366d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:02.240136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:02.240777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:02.241051 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:02.241321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:02.250387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:02.251822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:02.261153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1320/2644] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:02.550612 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:02.552199 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:02.566615 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821496f-edd7-4054-a97b-ad17b0c677fb None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:02.573208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:02.574098 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:02.574453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:02.574649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] PATH_INFO: `/v3/projects/c3f374b08ec944d19a382bef07d0345a` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:02.576015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f374b08ec944d19a382bef07d0345a)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:02.578117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:02.627915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f374b08ec944d19a382bef07d0345a', 'request_id': 'req-a03add5d-d641-4290-9563-14ccb7005d67'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.633247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f374b08ec944d19a382bef07d0345a', 'request_id': 'req-a03add5d-d641-4290-9563-14ccb7005d67'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.639658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f374b08ec944d19a382bef07d0345a', 'request_id': 'req-a03add5d-d641-4290-9563-14ccb7005d67'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.658533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] The token cache is being invalidate because project c3f374b08ec944d19a382bef07d0345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:02.658914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.659922 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.662370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1325/2645] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:02 2022] DELETE /identity/v3/projects/c3f374b08ec944d19a382bef07d0345a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:02.668309 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:02.678941 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e8b2-a693-4127-b69e-ed3afc075796 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:02.740547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:02.741450 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:02.741878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:02.742190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] PATH_INFO: `/v3/users/15da0edcaa9e4e098185b3345d2c9161` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:02.743723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15da0edcaa9e4e098185b3345d2c9161)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:02.745785 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:02.795987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15da0edcaa9e4e098185b3345d2c9161'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.811125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15da0edcaa9e4e098185b3345d2c9161'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.811929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15da0edcaa9e4e098185b3345d2c9161'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.819204 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15da0edcaa9e4e098185b3345d2c9161'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.828078 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-04aff15c-7865-449e-adea-929f3c444814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15da0edcaa9e4e098185b3345d2c9161'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:02.834233 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1321/2646] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:02 2022] DELETE /identity/v3/users/15da0edcaa9e4e098185b3345d2c9161 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:04:03.137508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.139957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:03.154131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03add5d-d641-4290-9563-14ccb7005d67 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.219504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.220453 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.220731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.221002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] PATH_INFO: `/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.222190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ca38bf26cf48928bad41a3f90ca2b9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.224363 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.277877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ca38bf26cf48928bad41a3f90ca2b9', 'request_id': 'req-3cb1c9a7-dcb2-41a8-a158-225da18f864a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:03.279215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:03.285569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ca38bf26cf48928bad41a3f90ca2b9', 'request_id': 'req-3cb1c9a7-dcb2-41a8-a158-225da18f864a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:03.292360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ca38bf26cf48928bad41a3f90ca2b9', 'request_id': 'req-3cb1c9a7-dcb2-41a8-a158-225da18f864a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:03.306486 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] The token cache is being invalidate because project 30ca38bf26cf48928bad41a3f90ca2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:03.306762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:03.307389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3cb1c9a7-dcb2-41a8-a158-225da18f864a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:03.309029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1326/2647] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:03 2022] DELETE /identity/v3/projects/30ca38bf26cf48928bad41a3f90ca2b9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:03.314328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.314827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.314924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.333716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:03.334955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:03.405387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1322/2648] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:03.413727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.414019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.414296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.434657 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:03.436252 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:03.445390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1327/2649] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:03.449779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.454990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:03.462435 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09797a-af89-41d8-9540-d50e85aa9e35 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.469169 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.469862 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.470143 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.470416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.471144 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:04:03.471448 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.473103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.485377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1323/2650] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:04:03.490185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.501210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416cbc-d863-4a67-8cff-4e54b722b5f0 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.507013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.507771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.508370 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.508662 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.509485 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.511462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.553773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1328/2651] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:04:03.558352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.571701 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ffc0f-74a0-42d1-9209-6295a40ebae0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.653686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.654375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.654655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.654929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.655757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.658764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.680270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1324/2652] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:04:03.685001 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.696990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b67b7-32f5-423d-9815-71a855c6b40d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.701583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.702360 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.702639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.702912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.703656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.705295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.715984 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1329/2653] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:03.720624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.730539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fccd86-ef18-4072-95fd-52f581b2c03c admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.735669 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.736355 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.736634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.736912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.737656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.739291 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2846721-9266-4740-9618-41d903deb57e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.751493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1325/2654] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:03.757883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:03.774257 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40f58b-690f-4876-a337-35a745aba0da admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:03.781750 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:03.782767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.783434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.783799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] PATH_INFO: `/v3/projects/ed87276cff744d23a6b6d1e71c914fe9/users/d49e40a1efb54279a615e0c48be88b90/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.800135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed87276cff744d23a6b6d1e71c914fe9, user_id=d49e40a1efb54279a615e0c48be88b90, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:03.802860 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:03.822672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1330/2655] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:03 2022] PUT /identity/v3/projects/ed87276cff744d23a6b6d1e71c914fe9/users/d49e40a1efb54279a615e0c48be88b90/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:03.838105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:03.838401 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:03.838687 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:03.867663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] MFA Rules not processed for user `d49e40a1efb54279a615e0c48be88b90`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:03.868916 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:03.945761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1326/2656] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:03.950458 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.015773 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d34c9-9384-4728-b62d-be34b363049d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.020914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.021593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.021874 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.022157 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.022856 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.024851 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.052457 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1331/2657] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:03 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:04.056880 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.068508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3ac70-2677-48cb-bca7-997eac70049d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.119246 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:04.128434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.129368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.129731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.130135 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.131040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.134381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.159838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1327/2658] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:04 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:04.163910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.177234 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6a13-2134-4c85-80a3-af4bde82ebb3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.183107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.183806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.184141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.184375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.185102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.186732 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.198899 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1332/2659] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:04.203171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.214501 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bae622-b64b-4100-bda4-eaa565180cd6 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.220251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.220953 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.221238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.221522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.222810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.224494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ded1a-2c91-4d30-846c-5fbe69e67172 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.237174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1328/2660] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:04.241713 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.253237 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ca2e6-5331-4d94-95e5-c34bf3bb8c08 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.258996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.259967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.260235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.260539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] PATH_INFO: `/v3/projects/b3917d5a6f9a42288dea0d08cf8f207c/users/9eb88fe1b47545e4b44ca835855ec23d/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.276099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3917d5a6f9a42288dea0d08cf8f207c, user_id=9eb88fe1b47545e4b44ca835855ec23d, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.278819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.294419 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1333/2661] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:04 2022] PUT /identity/v3/projects/b3917d5a6f9a42288dea0d08cf8f207c/users/9eb88fe1b47545e4b44ca835855ec23d/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:04.363799 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.364406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.364406 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.395512 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] MFA Rules not processed for user `9eb88fe1b47545e4b44ca835855ec23d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:04.396684 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:04.469827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1329/2662] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:04.474513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.503191 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:04.549526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2987a-79d4-4996-94bf-0bebc853662c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.554355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.555034 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.555318 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.555600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.556321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.558233 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.585722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1334/2663] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:04 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:04.591295 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.602721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23653975-8fd4-4449-af6d-d9cc7619e280 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.665440 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.666197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.666482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.666760 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.667475 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.669471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.693323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1330/2664] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:04 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:04.695229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.707284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c30df-3283-496a-9b45-5446ae16fe97 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.712446 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.713182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.713461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.713735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.714540 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.716235 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.727867 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1335/2665] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:04.732549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.743903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8dddf-9163-42b5-9c4c-54042a60501a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.749597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.751240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.751522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.751800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] PATH_INFO: `/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef/users/377b3c5767354dae94967ddb67ec78ee/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.768020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6014fc1b099420d97f9c68fd4e1a7ef, user_id=377b3c5767354dae94967ddb67ec78ee, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.770717 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.791651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1331/2666] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:04 2022] PUT /identity/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef/users/377b3c5767354dae94967ddb67ec78ee/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:04.797779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.810260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2ebcc-5421-4592-8c26-73a7e8899652 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.874198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.874981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.875263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.875539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.876296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.877942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01b8f-2c0c-461f-b396-1027cb05ec29 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.890912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1336/2667] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:04.892416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:04.904102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc4b0a-eda0-4c51-869e-565e373dcef5 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:04.910704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:04.911627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.911912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.912220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] PATH_INFO: `/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef/users/377b3c5767354dae94967ddb67ec78ee/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.914388 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6014fc1b099420d97f9c68fd4e1a7ef, user_id=377b3c5767354dae94967ddb67ec78ee, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:04.917262 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:04.929333 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1332/2668] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:04 2022] PUT /identity/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef/users/377b3c5767354dae94967ddb67ec78ee/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:04.940396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:04.940801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:04.941113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:04.971754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] MFA Rules not processed for user `377b3c5767354dae94967ddb67ec78ee`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:04.973036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:05.055785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1337/2669] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:05.068539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:05.071531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:05.101079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:05.141854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cfe45-4481-4300-ba88-e019265d3ba4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:05.145950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:05.146624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:05.146903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:05.147178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:05.206053 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:05.207436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:05.216778 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1333/2670] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:06.244312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:06.246315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:06.274508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:06.313037 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7b3c2-da88-4bb7-b7ea-c6d38335fe8f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:06.317014 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:06.317681 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:06.317956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:06.318229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:06.320205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:06.381878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:06.383324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:06.393105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1338/2671] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:06.758216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:06.768214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d968f-02f6-4934-930f-f9362337216a service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:06.772015 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:06.772661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:06.772929 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:06.773207 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:06.832987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:06.834424 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:06.843786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1334/2672] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:07.440042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:07.451938 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7656d-157b-4ee3-aac2-fa78ff3cec78 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:07.479091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:07.479781 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:07.480092 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:07.480400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:07.499776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:07.501283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:07.521649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1339/2673] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:11.455645 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:11.457183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:11.471072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2c9aa-0d9d-4d58-ae8a-b4bdeb39bbf4 service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:11.474731 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:11.475461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:11.475926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:11.476294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:11.478121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:11.552404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:11.554072 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f279e2-7a78-4b6c-bc7d-9be4873e7c34 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:11.566550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1335/2674] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:04:11.575522 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:11.586315 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86abf-118c-42f4-9312-60b0569b50df service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:11.590767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:11.591727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:11.592106 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:11.592444 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] PATH_INFO: `/v3/users/d49e40a1efb54279a615e0c48be88b90` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:11.593815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d49e40a1efb54279a615e0c48be88b90)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:11.595852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:11.646270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd49e40a1efb54279a615e0c48be88b90'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:11.659031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd49e40a1efb54279a615e0c48be88b90'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:11.659762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd49e40a1efb54279a615e0c48be88b90'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:11.667060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd49e40a1efb54279a615e0c48be88b90'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:11.676696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd49e40a1efb54279a615e0c48be88b90'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:11.683190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1340/2675] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:11 2022] DELETE /identity/v3/users/d49e40a1efb54279a615e0c48be88b90 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:04:11.687804 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:11.688234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:11.688520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:11.695922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:11.706360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:11.707712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:11.769300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1336/2676] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:11.780578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:11.782583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:11.821456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:11.848482 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8db59-3cdb-4dba-8229-eb3d78460cbf admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:11.852696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:11.853355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:11.853640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:11.854036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:11.864362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:11.865815 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:11.876957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1341/2677] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:12.171984 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:12.173800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:12.191694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9782-6b8a-4c03-8980-6a94e96099f9 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:12.197430 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:12.198691 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:12.199548 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:12.200148 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] PATH_INFO: `/v3/projects/ed87276cff744d23a6b6d1e71c914fe9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:12.202266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed87276cff744d23a6b6d1e71c914fe9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:12.205675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:12.262482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed87276cff744d23a6b6d1e71c914fe9', 'request_id': 'req-13ac80fc-89b2-4304-a35c-8d6306df9a29'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.269343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed87276cff744d23a6b6d1e71c914fe9', 'request_id': 'req-13ac80fc-89b2-4304-a35c-8d6306df9a29'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.275045 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed87276cff744d23a6b6d1e71c914fe9', 'request_id': 'req-13ac80fc-89b2-4304-a35c-8d6306df9a29'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.289408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] The token cache is being invalidate because project ed87276cff744d23a6b6d1e71c914fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:12.289693 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.290328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-13ac80fc-89b2-4304-a35c-8d6306df9a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.291861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1337/2678] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:12 2022] DELETE /identity/v3/projects/ed87276cff744d23a6b6d1e71c914fe9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:12.295979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:12.307194 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5e6fa-ee86-44ca-b03d-42074544fe11 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:12.369973 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:12.370854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:12.371132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:12.371406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] PATH_INFO: `/v3/users/9eb88fe1b47545e4b44ca835855ec23d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:12.372595 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb88fe1b47545e4b44ca835855ec23d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:12.374515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:12.426138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb88fe1b47545e4b44ca835855ec23d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.440042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb88fe1b47545e4b44ca835855ec23d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.440923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb88fe1b47545e4b44ca835855ec23d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.447801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb88fe1b47545e4b44ca835855ec23d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.456670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb88fe1b47545e4b44ca835855ec23d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:12.463805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1342/2679] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:12 2022] DELETE /identity/v3/users/9eb88fe1b47545e4b44ca835855ec23d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:04:12.878921 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:12.879456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:12.879456 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:12.900481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] MFA Rules not processed for user `0141317702184143b85a9dce65ca3b84`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:12.901831 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:12.974715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1338/2680] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:12.995359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:12.996906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:13.084468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:13.088624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064c748-daf4-478e-9666-2e0c21c9b61b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:13.098758 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:13.100071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:13.100497 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:13.100918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] PATH_INFO: `/v3/projects/b3917d5a6f9a42288dea0d08cf8f207c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:13.102570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3917d5a6f9a42288dea0d08cf8f207c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:13.104728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:13.188477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3917d5a6f9a42288dea0d08cf8f207c', 'request_id': 'req-11512dac-1265-48ba-b2cb-f5ba42d0dbde'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.209344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3917d5a6f9a42288dea0d08cf8f207c', 'request_id': 'req-11512dac-1265-48ba-b2cb-f5ba42d0dbde'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.223579 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3917d5a6f9a42288dea0d08cf8f207c', 'request_id': 'req-11512dac-1265-48ba-b2cb-f5ba42d0dbde'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.247701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] The token cache is being invalidate because project b3917d5a6f9a42288dea0d08cf8f207c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:13.248043 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.250351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.252390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1343/2681] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:12 2022] DELETE /identity/v3/projects/b3917d5a6f9a42288dea0d08cf8f207c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:13.256611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:13.269184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165a9b-3757-4e28-91b4-093ee495613b None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:13.318515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:13.355688 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:13.356608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:13.356963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:13.357296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] PATH_INFO: `/v3/users/377b3c5767354dae94967ddb67ec78ee` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:13.358662 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377b3c5767354dae94967ddb67ec78ee)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:13.361100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:13.432142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377b3c5767354dae94967ddb67ec78ee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.447565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377b3c5767354dae94967ddb67ec78ee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.448853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377b3c5767354dae94967ddb67ec78ee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.463765 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377b3c5767354dae94967ddb67ec78ee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.473954 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-d1653f2f-893b-41df-ad84-b116b210adc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377b3c5767354dae94967ddb67ec78ee'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:13.484303 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1339/2682] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:13 2022] DELETE /identity/v3/users/377b3c5767354dae94967ddb67ec78ee => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:13.987593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:13.989190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:14.017771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512dac-1265-48ba-b2cb-f5ba42d0dbde admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.142026 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.142951 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.143229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.143968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] PATH_INFO: `/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.147568 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6014fc1b099420d97f9c68fd4e1a7ef)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.150336 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.233678 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6014fc1b099420d97f9c68fd4e1a7ef', 'request_id': 'req-3e03b912-57e6-452d-9eb5-0d5b18db568c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:14.249374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6014fc1b099420d97f9c68fd4e1a7ef', 'request_id': 'req-3e03b912-57e6-452d-9eb5-0d5b18db568c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:14.261104 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6014fc1b099420d97f9c68fd4e1a7ef', 'request_id': 'req-3e03b912-57e6-452d-9eb5-0d5b18db568c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:14.285817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] The token cache is being invalidate because project e6014fc1b099420d97f9c68fd4e1a7ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:14.286166 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:14.286759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-3e03b912-57e6-452d-9eb5-0d5b18db568c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:14.291033 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1344/2683] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:13 2022] DELETE /identity/v3/projects/e6014fc1b099420d97f9c68fd4e1a7ef => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:14.296853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.297786 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.298302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.335055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:14.336961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:14.414423 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1340/2684] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:14.422917 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.423186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.423883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.446676 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:14.447969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:14.463101 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1345/2685] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:14.468024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.481250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133dbec-c206-4b0e-9e44-422d40851f9e None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.491777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.492646 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.494016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.494396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.498216 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:04:14.498878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.500963 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.520779 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1341/2686] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:04:14.524774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.540526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814347a-187b-47e3-8f5e-cb9ad47fd4fc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.545825 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.546620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.547004 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.547344 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.548421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.550450 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.556254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:14.585128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1346/2687] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:14 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:14.589354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.611612 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4423407-d0e0-4cc2-87c9-300da0e39b29 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.664087 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:14.697905 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.698845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.699126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.699408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.704318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.706580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.732056 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1342/2688] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:14 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:14.733890 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.745342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723714d-7ba7-4c57-a9a9-e1478be3cdba admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.750141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.750932 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.751273 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.751609 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.752725 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.754468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.765216 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1347/2689] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:04:14.769725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.781757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c6c39-7a04-487d-94e0-d231dbd2eb99 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.787181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.787950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.788235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.788517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.789263 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.790887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4b2fe-afa2-42a3-8371-458f635525bf admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.802317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1343/2690] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:14.806436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.816669 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff34773-a78a-412d-8e98-742516c13a00 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:14.821390 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:14.822302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.822583 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.822857 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] PATH_INFO: `/v3/projects/bf4b0509ba7049e9bcccf1231113c8d9/users/b594d2cb0f104dd1b8567928e6a15581/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.841062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4b0509ba7049e9bcccf1231113c8d9, user_id=b594d2cb0f104dd1b8567928e6a15581, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:14.842771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:14.856805 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1348/2691] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:14 2022] PUT /identity/v3/projects/bf4b0509ba7049e9bcccf1231113c8d9/users/b594d2cb0f104dd1b8567928e6a15581/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:14.862084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ea49-4cfe-4991-a824-c98b1444342e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:14.862380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ea49-4cfe-4991-a824-c98b1444342e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:14.862689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ea49-4cfe-4991-a824-c98b1444342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:14.884557 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-55d6ea49-4cfe-4991-a824-c98b1444342e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:14.885806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-55d6ea49-4cfe-4991-a824-c98b1444342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:14.966033 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1344/2692] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:04:14.973424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:14.975488 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:15.137473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b0fd-1759-49ca-94bd-a30f34592f7f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:15.142688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:15.143654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:15.143971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:15.144256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:15.165801 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:15.167538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:15.199619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1349/2693] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:22.117196 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.117886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.117886 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.147564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] MFA Rules not processed for user `b594d2cb0f104dd1b8567928e6a15581`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:22.148863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:22.270256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1345/2694] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:22.275993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.290247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf4561-7b6e-4502-ad06-987e17f628f3 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.296116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.296813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.297091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.297509 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.298248 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.300196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.328679 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1350/2695] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:22 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:22.332855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.349156 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a3d01-b917-4d09-97ea-544780d85ec8 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.412992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:22.430198 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.431035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.431595 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.432014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.433353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.436966 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.467089 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1346/2696] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:22 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:04:22.471264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.485759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc027-e6d8-461d-a78d-4cfe85b30ffc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.491262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.492322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.492802 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.492939 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.493649 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.495294 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.508182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1351/2697] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:22.512317 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.523032 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37debc11-9c29-4b1b-b36f-340af1fd441b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.527816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.528821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.529084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.529363 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] PATH_INFO: `/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b/users/18ab046741ff49bca2578918128ee02c/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.556000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17978e22db0c4ed68cd50fbfb5821e6b, user_id=18ab046741ff49bca2578918128ee02c, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.558762 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.578437 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1347/2698] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:22 2022] PUT /identity/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b/users/18ab046741ff49bca2578918128ee02c/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:22.582872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.593563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91efd7-fabf-47fa-a38d-a40ad0aecb71 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.665417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.666111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.666408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.666627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.667374 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.669052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.684109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1352/2699] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:22.685372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:22.696353 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50964b2c-40a2-4cfb-a19c-6a356dda46a0 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:22.701191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:22.702178 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:22.702536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:22.703019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] PATH_INFO: `/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b/users/18ab046741ff49bca2578918128ee02c/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:22.705234 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17978e22db0c4ed68cd50fbfb5821e6b, user_id=18ab046741ff49bca2578918128ee02c, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:22.708055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:22.719044 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1348/2700] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:22 2022] PUT /identity/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b/users/18ab046741ff49bca2578918128ee02c/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:23.142003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:23.144915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:23.182513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772d31d-a8a6-4566-9e6c-4ef91b88f008 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:23.203814 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:23.316916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:23.318513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:23.318795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:23.319067 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] PATH_INFO: `/v3/users/600fb1135fe040e2aa1c03628da8bf93` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:23.321644 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] RBAC: Authorizing `identity:delete_user(user_id=600fb1135fe040e2aa1c03628da8bf93)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:23.323785 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:23.593479 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600fb1135fe040e2aa1c03628da8bf93'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:23.610388 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600fb1135fe040e2aa1c03628da8bf93'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:23.611701 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600fb1135fe040e2aa1c03628da8bf93'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:23.624816 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600fb1135fe040e2aa1c03628da8bf93'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:23.638013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600fb1135fe040e2aa1c03628da8bf93'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:23.646923 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1353/2701] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:23 2022] DELETE /identity/v3/users/600fb1135fe040e2aa1c03628da8bf93 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:04:23.933647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:23.936900 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:23.971791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca5548-8ca4-48e6-bbff-71eda24d2047 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:24.073706 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:24.074597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:24.074881 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:24.075160 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] PATH_INFO: `/v3/projects/3d2953bf079541b28189cc9ad88ad753` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:24.076553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2953bf079541b28189cc9ad88ad753)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:24.078494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:24.191124 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2953bf079541b28189cc9ad88ad753', 'request_id': 'req-3e8c8540-62e0-4d21-a396-04a2e87ef6be'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.202570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2953bf079541b28189cc9ad88ad753', 'request_id': 'req-3e8c8540-62e0-4d21-a396-04a2e87ef6be'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.219167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2953bf079541b28189cc9ad88ad753', 'request_id': 'req-3e8c8540-62e0-4d21-a396-04a2e87ef6be'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.246004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] The token cache is being invalidate because project 3d2953bf079541b28189cc9ad88ad753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:24.246611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.247545 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.249413 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1349/2702] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:23 2022] DELETE /identity/v3/projects/3d2953bf079541b28189cc9ad88ad753 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:24.254047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:24.265062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901d04-cc56-4d1c-b08e-ae8f2744507d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:24.357355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:24.359387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:24.359912 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:24.360588 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] PATH_INFO: `/v3/users/388af5ed115a40038c0674b1cba10dc2` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:24.362212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388af5ed115a40038c0674b1cba10dc2)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:24.364383 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:24.448469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:24.463652 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388af5ed115a40038c0674b1cba10dc2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.491107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388af5ed115a40038c0674b1cba10dc2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.491889 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388af5ed115a40038c0674b1cba10dc2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.504326 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388af5ed115a40038c0674b1cba10dc2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.525327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-db6da50b-2317-489d-a1db-4e2f1f173d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388af5ed115a40038c0674b1cba10dc2'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:24.533231 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1354/2703] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:24 2022] DELETE /identity/v3/users/388af5ed115a40038c0674b1cba10dc2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:24.852408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:24.854134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:24.871334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8540-62e0-4d21-a396-04a2e87ef6be admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:24.921191 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:24.979922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:24.981663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:24.982102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:24.982967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] PATH_INFO: `/v3/projects/4100a1d9888c420c9e36353196577e3b` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:24.985490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4100a1d9888c420c9e36353196577e3b)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:24.987975 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.086276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4100a1d9888c420c9e36353196577e3b', 'request_id': 'req-377d2ce9-b61f-48b9-8bf2-623fbae41131'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:25.107334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4100a1d9888c420c9e36353196577e3b', 'request_id': 'req-377d2ce9-b61f-48b9-8bf2-623fbae41131'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:25.115601 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4100a1d9888c420c9e36353196577e3b', 'request_id': 'req-377d2ce9-b61f-48b9-8bf2-623fbae41131'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:25.143071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] The token cache is being invalidate because project 4100a1d9888c420c9e36353196577e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:25.143629 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:25.144754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-377d2ce9-b61f-48b9-8bf2-623fbae41131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:25.146508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1350/2704] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:24 2022] DELETE /identity/v3/projects/4100a1d9888c420c9e36353196577e3b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:25.151677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.152256 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.152624 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.181618 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:25.183719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:25.265451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1355/2705] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:25.274505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.275389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.275978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.313031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:25.316877 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:25.329378 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1351/2706] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:25.331403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.342670 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa7cbf-2a26-411b-8fb1-56d4d171bc8c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.348425 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.349397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.349687 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.349966 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.350717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:04:25.351027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.352873 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.370314 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1356/2707] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:04:25.374389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.393258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa204fc-e91d-4f06-9469-84bae00c6801 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.398549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.399240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.399526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.399806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.400546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.403681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.439356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1352/2708] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:25 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:25.443102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.454826 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2e8b4-0cee-450b-93bd-62adbfe9f242 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.541091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.541849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.542132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.542404 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.543121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.545317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.574416 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1357/2709] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:25 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:04:25.575883 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.589299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dd63b-f39b-47df-9439-54e1ac1995fe admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.596034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.597235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.597235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.597463 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.598063 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.599727 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.639546 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1353/2710] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:04:25.643648 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.654900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288b3c1-fc2f-4ae3-bea2-32d8e536c9da admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.660432 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.661121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.661402 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.661683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.662593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.664032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17f40f-8d7c-4ad8-a89b-ae9c61802702 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.678002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1358/2711] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:25.682120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.697854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a9fed-6669-436a-8e30-1cb147f1e350 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:25.703937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:25.706036 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.706543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.707029 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] PATH_INFO: `/v3/projects/fea9c5d098d646b89213d35f737e2308/users/2e2b4b08c1b34a7f86996a806953ac1d/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.726375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea9c5d098d646b89213d35f737e2308, user_id=2e2b4b08c1b34a7f86996a806953ac1d, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:25.729851 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:25.765194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1354/2712] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:25 2022] PUT /identity/v3/projects/fea9c5d098d646b89213d35f737e2308/users/2e2b4b08c1b34a7f86996a806953ac1d/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:25.770208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69568c1d-afd0-454f-a40d-768fad89ac03 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:25.770607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69568c1d-afd0-454f-a40d-768fad89ac03 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:25.771006 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69568c1d-afd0-454f-a40d-768fad89ac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:25.804175 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-69568c1d-afd0-454f-a40d-768fad89ac03 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:25.805581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-69568c1d-afd0-454f-a40d-768fad89ac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:25.839362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:25.906039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1359/2713] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:25.928362 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:25.930743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:26.015515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5955a8-0026-4f75-a456-3dac698da605 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:26.020167 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:26.020842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:26.021129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:26.021408 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:26.032735 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:26.034577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:26.041508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:26.052494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1355/2714] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:28.997109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e21f8-1307-459c-b9ff-09e8eb830064 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:28.997710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e21f8-1307-459c-b9ff-09e8eb830064 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:28.997710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e21f8-1307-459c-b9ff-09e8eb830064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:29.052389 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-891e21f8-1307-459c-b9ff-09e8eb830064 None None] MFA Rules not processed for user `18ab046741ff49bca2578918128ee02c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:29.054055 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-891e21f8-1307-459c-b9ff-09e8eb830064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:29.152764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1360/2715] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:29.167798 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:29.169857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:29.291922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae357e-ab5a-4488-ba6b-db1db147061a service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:29.302705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44cc172ac2904c84a84c7f4dbb707901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:29.303759 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:29.305030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:29.305374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:29.312704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:29.422332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:29.424627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:29.439138 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1356/2716] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:04:29.584312 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15cf2-d1ec-4c6f-9c3f-59560ec9da2c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:29.584797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15cf2-d1ec-4c6f-9c3f-59560ec9da2c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:29.585230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15cf2-d1ec-4c6f-9c3f-59560ec9da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:29.613046 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-a0c15cf2-d1ec-4c6f-9c3f-59560ec9da2c None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:29.615224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-a0c15cf2-d1ec-4c6f-9c3f-59560ec9da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:29.715561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1361/2717] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:29.827100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:29.830876 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:30.005426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccbdff-c6ae-46fc-b322-fb767c281fe3 service glance] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:30.012843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:30.014170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:30.014675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:30.015815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:30.027789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:30.030200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:30.042462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1357/2718] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:04:32.627396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:32.628376 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:32.628759 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:32.696631 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] MFA Rules not processed for user `2e2b4b08c1b34a7f86996a806953ac1d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:32.697962 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:32.776571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:32.931008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1362/2719] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:32.940937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.075368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:33.083034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e26f2-c706-4826-b735-bae2ea9f570b service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.092864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.098443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.098443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.098443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.098443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.110125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.216705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1358/2720] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:04:32 2022] POST /identity/v3/projects => generated 379 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:04:33.220442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.235909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7ef58-739e-4aa1-b42f-875c90fb3a7a None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.400877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.401591 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.401876 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.402135 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.402852 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.404847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.482855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1363/2721] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:04:33 2022] POST /identity/v3/users => generated 296 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:04:33.486737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.512821 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83dd3de-6d15-4032-bbbc-aa7de45eb2b3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.526652 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.527569 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.528043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.529277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.530197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.532297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.594114 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.605244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1359/2722] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:33.608842 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838999cf-b136-44ac-8463-4167d7877950 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.623563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.624515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.624792 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.625064 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] PATH_INFO: `/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d/users/d2a44e619a294756ae6ab8a105ac64d8/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.681721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b994fb844e01431ea8e49c1a5d0ad61d, user_id=d2a44e619a294756ae6ab8a105ac64d8, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.684493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.713677 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1364/2723] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:33 2022] PUT /identity/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d/users/d2a44e619a294756ae6ab8a105ac64d8/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:04:33.717728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.733280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a44281-b41a-49a5-8a51-e1a29eac742d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.866633 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.867332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.867611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.867906 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.868663 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.870296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a548ce0-8572-44a4-b88b-3d2ff9e8af83 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.892241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1360/2724] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:04:33.893552 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:33.906091 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44ea1-116e-467d-a61f-5ac84d55531d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:33.913468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:33.914614 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:33.915063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:33.915469 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] PATH_INFO: `/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d/users/d2a44e619a294756ae6ab8a105ac64d8/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:33.919978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b994fb844e01431ea8e49c1a5d0ad61d, user_id=d2a44e619a294756ae6ab8a105ac64d8, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:33.924325 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:33.938548 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1365/2725] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:04:33 2022] PUT /identity/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d/users/d2a44e619a294756ae6ab8a105ac64d8/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:40.985162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d679-8c51-4705-8161-4ef821ba574e None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:40.985747 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d679-8c51-4705-8161-4ef821ba574e None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:40.985860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d679-8c51-4705-8161-4ef821ba574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:41.027220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-03d4d679-8c51-4705-8161-4ef821ba574e None None] MFA Rules not processed for user `d2a44e619a294756ae6ab8a105ac64d8`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:41.028483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-03d4d679-8c51-4705-8161-4ef821ba574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:41.132257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1361/2726] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:41.150152 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:41.153099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:41.281145 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:41.347408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea0943-ade5-4cc7-96f8-952333d6d25d admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:41.354278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:41.355172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:41.355572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:41.356007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:41.438036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:41.440308 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13848ba4-319b-4bbe-adee-3fca6d273b98 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:41.465771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1366/2727] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:04:46.713173 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:46.713754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:46.713861 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:46.736690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] MFA Rules not processed for user `b594d2cb0f104dd1b8567928e6a15581`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:46.737840 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:46.798407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:46.826576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1362/2728] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:49.118827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:49.120027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:49.120490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:49.173057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:04:49.174560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:49.276562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1367/2729] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Jul 14 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:04:49.291849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:49.417094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408b1dd-134f-48d2-a479-45b5e8dc9198 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:49.442013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:49.452825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:49.453330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:49.455335 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:49.473074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:49.475649 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:49.498326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1363/2730] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:04:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:04:57.455275 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:57.457115 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:04:57.528747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd6712-f4ce-43a2-ba6c-60a57b709e27 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:57.534373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:57.535392 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:57.536138 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:57.536418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] PATH_INFO: `/v3/users/75a9903b3597458f8f5af4d06cd07d4c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:57.537756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a9903b3597458f8f5af4d06cd07d4c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:57.539714 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:57.541532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:04:57.626291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a9903b3597458f8f5af4d06cd07d4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:57.660029 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a9903b3597458f8f5af4d06cd07d4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:57.663176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a9903b3597458f8f5af4d06cd07d4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:57.678671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a9903b3597458f8f5af4d06cd07d4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:57.688671 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-47743130-0c7c-4ef7-83ad-0268a796b654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a9903b3597458f8f5af4d06cd07d4c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:57.698466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1368/2731] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:04:57 2022] DELETE /identity/v3/users/75a9903b3597458f8f5af4d06cd07d4c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:04:57.932134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:04:57.944812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969fc9b-f7cc-4229-8e23-e36c845a1253 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:04:58.013912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:04:58.014942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:04:58.015296 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:04:58.015640 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] PATH_INFO: `/v3/projects/ad57e412dc1e46c39f1121f3e4ca8ee9` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:04:58.016993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad57e412dc1e46c39f1121f3e4ca8ee9)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:04:58.019174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:04:58.070439 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad57e412dc1e46c39f1121f3e4ca8ee9', 'request_id': 'req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:58.077332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad57e412dc1e46c39f1121f3e4ca8ee9', 'request_id': 'req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:58.084455 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad57e412dc1e46c39f1121f3e4ca8ee9', 'request_id': 'req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:58.099656 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] The token cache is being invalidate because project ad57e412dc1e46c39f1121f3e4ca8ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:04:58.100079 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:58.101026 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:04:58.102819 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1364/2732] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:04:57 2022] DELETE /identity/v3/projects/ad57e412dc1e46c39f1121f3e4ca8ee9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:05:01.275709 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10da2-da98-402d-b81e-5602b26621de None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:01.276324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10da2-da98-402d-b81e-5602b26621de None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:01.276324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10da2-da98-402d-b81e-5602b26621de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:01.293999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-89e10da2-da98-402d-b81e-5602b26621de None None] MFA Rules not processed for user `2e2b4b08c1b34a7f86996a806953ac1d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:01.295236 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-89e10da2-da98-402d-b81e-5602b26621de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:01.350866 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1369/2733] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:08.095506 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:11.343824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:16.018316 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:16.020081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:16.125016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ace60a-c9fc-4527-972a-1d1df4d40b9e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:16.128752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:16.129539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:16.129938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:16.130212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:16.132580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:16.210386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:16.211856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f056487-db6e-4fd6-9eb8-49518a830cd9 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:16.223860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1365/2734] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:20.433607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e10da2-da98-402d-b81e-5602b26621de None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:20.436223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-89e10da2-da98-402d-b81e-5602b26621de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:20.451080 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e10da2-da98-402d-b81e-5602b26621de None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:20.456084 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:20.470105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:20.470452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:20.470675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:20.473020 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:20.602504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:20.605983 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:20.622828 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1370/2735] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:24.236937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6143a-d3a8-40aa-8694-348cb594adbc None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:24.237511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6143a-d3a8-40aa-8694-348cb594adbc None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:24.237511 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6143a-d3a8-40aa-8694-348cb594adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:24.257171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-4de6143a-d3a8-40aa-8694-348cb594adbc None None] MFA Rules not processed for user `0141317702184143b85a9dce65ca3b84`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:05:24.258445 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4de6143a-d3a8-40aa-8694-348cb594adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:24.322240 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1366/2736] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:05:30.615997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=30 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:34.315200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:05:38.892477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:05:38.893971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:38.993268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1652a62-307b-44c0-9ef4-e0e176df070b service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:05:38.999347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:05:39.000173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:05:39.000471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:05:39.000745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:05:39.002653 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:05:39.014613 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:05:39.016253 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:05:39.026471 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1371/2737] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:05:49.022277 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=41 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:06:28.532563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:28.533134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:28.533134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:28.554341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] MFA Rules not processed for user `0141317702184143b85a9dce65ca3b84`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:06:28.556235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:28.565998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1367/2738] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:06:33.968832 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:33.983620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27730-4019-4f11-afa1-5648c5dc06ce service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:33.988358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:33.989002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:33.989286 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:33.989545 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:33.991722 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:34.065834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:34.067259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7d395-207c-438d-a44d-ee8244322c18 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:34.078629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1372/2739] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:06:35.279857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:35.290536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ede6c6-8a82-470a-ba5a-6ea1ef7cabc1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:35.294570 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:35.295212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:35.295484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:35.295922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:35.297589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:35.311241 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:35.312698 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:35.323702 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1368/2740] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:06:44.073461 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=34 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:06:45.319222 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=9 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:06:49.520141 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:49.520602 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:49.520710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:49.538223 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] MFA Rules not processed for user `8728607736914f729f751bb1d47bbca9`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:06:49.539417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:49.604250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1373/2741] 10.222.0.27 () {58 vars in 967 bytes} [Thu Jul 14 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:06:49.644324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:49.646008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:49.718930 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5ee62-bd5b-45ca-89e3-9ce53ff312b3 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:49.724508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f958e0eb8444871851792cb254b56a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:49.725150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:49.725428 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:49.725703 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:49.734152 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:49.735769 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:49.743737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1369/2742] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Jul 14 11:06:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 11:06:53.055076 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:53.071204 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae053ff-b5bf-4b33-9b3b-6bef1279ae6e None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:53.076459 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:53.077224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:53.077564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:53.077906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:53.079736 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:53.091535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:53.093507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:53.103942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1374/2743] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:06:56.347510 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:56.349042 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:56.411484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4a48b-c557-4d95-a069-68b8c0d577c9 service swift] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:56.416502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:56.417374 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:56.417653 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:56.417924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] PATH_INFO: `/v3/users/b594d2cb0f104dd1b8567928e6a15581` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:56.419095 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b594d2cb0f104dd1b8567928e6a15581)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:56.421370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:56.467334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b594d2cb0f104dd1b8567928e6a15581'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.478894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b594d2cb0f104dd1b8567928e6a15581'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.479553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b594d2cb0f104dd1b8567928e6a15581'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.486371 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b594d2cb0f104dd1b8567928e6a15581'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.494214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b594d2cb0f104dd1b8567928e6a15581'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.500431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1370/2744] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:06:56 2022] DELETE /identity/v3/users/b594d2cb0f104dd1b8567928e6a15581 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:06:56.704740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:56.716872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd74c-6725-4890-aaac-a4b8ea8e0dda service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:56.779169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:56.780172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:56.780452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:56.780846 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] PATH_INFO: `/v3/projects/bf4b0509ba7049e9bcccf1231113c8d9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:56.782044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4b0509ba7049e9bcccf1231113c8d9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:56.784195 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:56.832047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4b0509ba7049e9bcccf1231113c8d9', 'request_id': 'req-949eb330-4382-450a-b8af-e76fc594bc0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.837366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4b0509ba7049e9bcccf1231113c8d9', 'request_id': 'req-949eb330-4382-450a-b8af-e76fc594bc0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.843518 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4b0509ba7049e9bcccf1231113c8d9', 'request_id': 'req-949eb330-4382-450a-b8af-e76fc594bc0a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.856632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] The token cache is being invalidate because project bf4b0509ba7049e9bcccf1231113c8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:06:56.856909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.857543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:56.859139 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1375/2745] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:06:56 2022] DELETE /identity/v3/projects/bf4b0509ba7049e9bcccf1231113c8d9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:06:56.863146 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:56.864825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:56.878563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58003f3a-debb-4928-bded-6c57b8e5501a admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:56.883525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:06:56.946238 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:56.947082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:56.947410 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:56.947695 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] PATH_INFO: `/v3/users/18ab046741ff49bca2578918128ee02c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:56.949086 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ab046741ff49bca2578918128ee02c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:56.951183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:57.001375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ab046741ff49bca2578918128ee02c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.013082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ab046741ff49bca2578918128ee02c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.013764 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ab046741ff49bca2578918128ee02c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.021166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ab046741ff49bca2578918128ee02c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.031326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-6c433f0b-27bf-4163-82c0-8bf972009b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ab046741ff49bca2578918128ee02c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.037351 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1371/2746] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:06:56 2022] DELETE /identity/v3/users/18ab046741ff49bca2578918128ee02c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:06:57.220790 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:06:57.222333 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:06:57.237948 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949eb330-4382-450a-b8af-e76fc594bc0a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:06:57.267052 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:06:57.308153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:06:57.309012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:06:57.309285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:06:57.309553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] PATH_INFO: `/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:06:57.310755 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17978e22db0c4ed68cd50fbfb5821e6b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:06:57.312707 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:06:57.368185 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17978e22db0c4ed68cd50fbfb5821e6b', 'request_id': 'req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.382658 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17978e22db0c4ed68cd50fbfb5821e6b', 'request_id': 'req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.389272 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17978e22db0c4ed68cd50fbfb5821e6b', 'request_id': 'req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.405442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] The token cache is being invalidate because project 17978e22db0c4ed68cd50fbfb5821e6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:06:57.405764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.406418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:06:57.408109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1376/2747] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:06:57 2022] DELETE /identity/v3/projects/17978e22db0c4ed68cd50fbfb5821e6b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:07.034052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:07.402481 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:24.411505 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:24.412034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:24.412132 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:24.430319 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] MFA Rules not processed for user `2e2b4b08c1b34a7f86996a806953ac1d`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:24.431518 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:24.487324 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1372/2748] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:25.286146 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:25.287664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:25.375284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0b7f8-5813-4cb6-a44b-4cec876e254a admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:25.380387 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f7505-13e1-42cd-895a-678226098379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:25.381049 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7505-13e1-42cd-895a-678226098379 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:25.381456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7505-13e1-42cd-895a-678226098379 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:25.381740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7505-13e1-42cd-895a-678226098379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:25.383507 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-577f7505-13e1-42cd-895a-678226098379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:25.395412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f7505-13e1-42cd-895a-678226098379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:25.397060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f7505-13e1-42cd-895a-678226098379 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:25.407839 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1377/2749] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:28.770399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:28.771806 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.841614 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac77730-ebbd-43c9-bc7c-c2b1eaefebf2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:28.845532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:28.846256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:28.846750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:28.847677 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:28.849835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:28.864992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:28.866723 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:28.876896 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1373/2750] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:35.402905 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=32 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:36.747510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f7505-13e1-42cd-895a-678226098379 service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:36.749119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-577f7505-13e1-42cd-895a-678226098379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:36.814996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f7505-13e1-42cd-895a-678226098379 service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:36.819470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:36.820379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:36.820663 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:36.820937 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] PATH_INFO: `/v3/users/0141317702184143b85a9dce65ca3b84` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:36.822117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0141317702184143b85a9dce65ca3b84)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:36.824053 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:36.874164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0141317702184143b85a9dce65ca3b84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.885871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0141317702184143b85a9dce65ca3b84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.886521 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0141317702184143b85a9dce65ca3b84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.895335 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0141317702184143b85a9dce65ca3b84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.905202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f393b42b-d707-42ba-a893-7da84e657313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0141317702184143b85a9dce65ca3b84'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:36.914675 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1378/2751] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:36 2022] DELETE /identity/v3/users/0141317702184143b85a9dce65ca3b84 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:37.096285 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.110489 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624ea58-83b3-4105-81a6-05c8cf51a171 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.177000 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.178010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.178293 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.178565 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] PATH_INFO: `/v3/projects/1bf3e1149d124a62b0984ec2598db78e` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.179775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf3e1149d124a62b0984ec2598db78e)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.181837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.231715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf3e1149d124a62b0984ec2598db78e', 'request_id': 'req-de0ffef4-22c9-4d18-a983-49d67a8bbc10'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:37.233192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:37.241878 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf3e1149d124a62b0984ec2598db78e', 'request_id': 'req-de0ffef4-22c9-4d18-a983-49d67a8bbc10'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:37.252141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf3e1149d124a62b0984ec2598db78e', 'request_id': 'req-de0ffef4-22c9-4d18-a983-49d67a8bbc10'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:37.268910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] The token cache is being invalidate because project 1bf3e1149d124a62b0984ec2598db78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:37.269197 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:37.269843 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-de0ffef4-22c9-4d18-a983-49d67a8bbc10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:37.271414 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1374/2752] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:37 2022] DELETE /identity/v3/projects/1bf3e1149d124a62b0984ec2598db78e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:37.276525 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.276810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.277105 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.339661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:37.340911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.386954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:37.403926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1379/2753] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:37.411726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.412049 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.412329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.430699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:37.432008 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.440904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1375/2754] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:37.445704 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.455516 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0458d-9aa6-430e-ada1-3bc3482b3ae7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.460203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.460848 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.461144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.461400 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] PATH_INFO: `/v3/domains` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.462283 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95327) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:37.462427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.464343 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.477224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1380/2755] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:37.481586 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.491436 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e320b6e-aee3-4914-90bd-b58fb3b001bd None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.496712 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.497381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.497660 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.497931 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.498625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.500647 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.545827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1376/2756] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:37.547097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.558177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fa15d-30d5-4e82-a616-f9983ab67c43 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.619044 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.619971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.620397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.620734 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.621711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.624803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.650682 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1381/2757] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:37.655174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.666208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c199b44-1936-4e0a-a74b-de742f9dfa77 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.671541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.672223 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.672480 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.672767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.673725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.675322 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.685589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1377/2758] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:37.689761 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.700057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2c21-022a-4f71-b141-b7d07534e9f3 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.705354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.705999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.706284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.706562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.707290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.708927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802174f-6469-4418-9ea7-613d677a4fcc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.720635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1382/2759] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:37.726805 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.739996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aa152-1270-42c8-b923-f29f7cde0072 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.745129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.746131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.746542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.746915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] PATH_INFO: `/v3/projects/86e45cd6b94949118f8126206af5d28d/users/0111f0b8284740e894df5e3b896f777b/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.763853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e45cd6b94949118f8126206af5d28d, user_id=0111f0b8284740e894df5e3b896f777b, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.766541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.782122 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1378/2760] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:37 2022] PUT /identity/v3/projects/86e45cd6b94949118f8126206af5d28d/users/0111f0b8284740e894df5e3b896f777b/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:37.788798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a04d-5830-449a-a002-a9bf7deaef9e None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.789313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a04d-5830-449a-a002-a9bf7deaef9e None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.789498 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a04d-5830-449a-a002-a9bf7deaef9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.809161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-6b19a04d-5830-449a-a002-a9bf7deaef9e None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:37.810753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-6b19a04d-5830-449a-a002-a9bf7deaef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.877048 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1383/2761] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:37.887052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:37.888922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:37.958010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d99f-1d26-4d6d-b77f-715d489fb630 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:37.961973 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44674880-2943-4423-991c-2c001588135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:37.962658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44674880-2943-4423-991c-2c001588135d service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:37.962943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44674880-2943-4423-991c-2c001588135d service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:37.963248 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44674880-2943-4423-991c-2c001588135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:37.973497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44674880-2943-4423-991c-2c001588135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:37.975012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44674880-2943-4423-991c-2c001588135d service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:37.984924 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1379/2762] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:38.623396 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.623708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.624019 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.651576 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] MFA Rules not processed for user `0111f0b8284740e894df5e3b896f777b`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:38.652835 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:38.699968 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:38.722817 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1384/2763] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:38.727276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44674880-2943-4423-991c-2c001588135d service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.736987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44674880-2943-4423-991c-2c001588135d service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.741903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.742468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.742756 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.742996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] PATH_INFO: `/v3/projects` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.743752 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.745672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.773690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1380/2764] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:38.777969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.791911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed199f-83b2-4857-bbde-572ed1618e8d None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.856456 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.857128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.857412 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.857686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] PATH_INFO: `/v3/users` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.858369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.860451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.882015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1385/2765] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:38 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:38.886210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.897622 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a85c-1931-43ef-bcb4-b001a517073e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.903352 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.904047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.904330 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.904604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.905326 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.906943 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.920405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1381/2766] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:38.925110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.935351 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964421-f31d-4f5f-89b9-bb9ca540ebb2 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:38.940735 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:38.941659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:38.941942 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:38.942225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] PATH_INFO: `/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b/users/a1dbef6436ce4e7983554b84c718fe77/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:38.957776 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddd16bd26fd460cb6f6a28566a48a0b, user_id=a1dbef6436ce4e7983554b84c718fe77, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:38.960605 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:38.975834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1386/2767] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:38 2022] PUT /identity/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b/users/a1dbef6436ce4e7983554b84c718fe77/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:38.980046 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:38.990417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb02ffa-d739-43b3-9625-3a3e30e8fd91 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.022386 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:39.051431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.052128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.052405 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.052674 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.053400 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.055011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.065349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1382/2768] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:39.069665 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.080063 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82667f-00c0-4bf6-8ece-127606a30029 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.084934 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.085821 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.086132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.086401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] PATH_INFO: `/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b/users/a1dbef6436ce4e7983554b84c718fe77/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.088510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddd16bd26fd460cb6f6a28566a48a0b, user_id=a1dbef6436ce4e7983554b84c718fe77, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.091102 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d038bf2-6017-4505-9098-4d5bde758d85 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.102008 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1387/2769] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:39 2022] PUT /identity/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b/users/a1dbef6436ce4e7983554b84c718fe77/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:39.565462 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:39.568498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.661120 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f11ba0-c3e8-429a-b2ee-e5000f9df69d admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:39.727547 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:39.728253 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.728494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.728766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.730745 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.742927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:39.744558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:39.755519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1383/2770] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 11:07:39.860408 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:39.861650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:39.861650 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:39.890827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] MFA Rules not processed for user `a1dbef6436ce4e7983554b84c718fe77`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:39.892210 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:39.982116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1388/2771] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:41.350899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.416496 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=100 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:41.422902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008a6e0-d94c-4eb5-a443-27933946e02b service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.429071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.429951 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.430228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.430502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] PATH_INFO: `/v3/users/0111f0b8284740e894df5e3b896f777b` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.431807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0111f0b8284740e894df5e3b896f777b)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.433797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.486673 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0111f0b8284740e894df5e3b896f777b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.499526 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0111f0b8284740e894df5e3b896f777b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.500289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0111f0b8284740e894df5e3b896f777b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.507208 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0111f0b8284740e894df5e3b896f777b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.516737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0111f0b8284740e894df5e3b896f777b'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.523331 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1384/2772] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:41 2022] DELETE /identity/v3/users/0111f0b8284740e894df5e3b896f777b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:41.720995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.732198 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd8c89-d26a-47c2-a7e9-8fce5b1b48f3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.752743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:41.799647 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.800537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.800819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.801113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] PATH_INFO: `/v3/projects/86e45cd6b94949118f8126206af5d28d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.802359 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e45cd6b94949118f8126206af5d28d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.804302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:41.860537 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e45cd6b94949118f8126206af5d28d', 'request_id': 'req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.867165 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e45cd6b94949118f8126206af5d28d', 'request_id': 'req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.874212 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e45cd6b94949118f8126206af5d28d', 'request_id': 'req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.889405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] The token cache is being invalidate because project 86e45cd6b94949118f8126206af5d28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:41.889691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.890348 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:41.892041 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1389/2773] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:41 2022] DELETE /identity/v3/projects/86e45cd6b94949118f8126206af5d28d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:41.898093 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:41.899617 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:41.913781 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aeaef-e53e-4150-a36d-09c20490c433 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:41.973842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:41.974725 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:41.975012 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:41.975415 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] PATH_INFO: `/v3/users/a1dbef6436ce4e7983554b84c718fe77` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:41.976519 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1dbef6436ce4e7983554b84c718fe77)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:41.978668 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.027104 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1dbef6436ce4e7983554b84c718fe77'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.039276 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1dbef6436ce4e7983554b84c718fe77'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.040017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1dbef6436ce4e7983554b84c718fe77'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.047170 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1dbef6436ce4e7983554b84c718fe77'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.049989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:42.060349 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-861c4846-2833-4622-885b-f3d9f3f588fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1dbef6436ce4e7983554b84c718fe77'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.068522 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1385/2774] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:07:41 2022] DELETE /identity/v3/users/a1dbef6436ce4e7983554b84c718fe77 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:42.262367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.264211 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.279607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3b17c-b497-46fd-a5b4-fbc16ae37730 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.341095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.341958 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.342263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.342539 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] PATH_INFO: `/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.343847 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddd16bd26fd460cb6f6a28566a48a0b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.346038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.394111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddd16bd26fd460cb6f6a28566a48a0b', 'request_id': 'req-c55ec496-f869-4434-85a0-e5538b10ea53'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.398089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:42.400543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddd16bd26fd460cb6f6a28566a48a0b', 'request_id': 'req-c55ec496-f869-4434-85a0-e5538b10ea53'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.409504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddd16bd26fd460cb6f6a28566a48a0b', 'request_id': 'req-c55ec496-f869-4434-85a0-e5538b10ea53'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.424728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] The token cache is being invalidate because project bddd16bd26fd460cb6f6a28566a48a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:07:42.425015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.425629 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c55ec496-f869-4434-85a0-e5538b10ea53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:07:42.427187 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1390/2775] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:07:42 2022] DELETE /identity/v3/projects/bddd16bd26fd460cb6f6a28566a48a0b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:42.432741 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.433105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.433105 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.451994 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:42.453190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.515426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1386/2776] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:42.525572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.525928 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.526270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.546538 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:42.547855 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.557967 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1391/2777] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:42.562490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.573232 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f7a77-4a0b-4345-9b7f-5513cc6ba200 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.578605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.579272 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.579997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.579997 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.580670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:07:42.580998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.582672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.594531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1387/2778] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:07:42.599247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.608451 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b40a88-2c3e-4cb7-9718-29b377924ef3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.613896 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.614635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.615039 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.615327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.616083 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.618329 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.645322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1392/2779] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:42.649311 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.659826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311ef5-b807-4166-afb3-7d7f5f2f33ff admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.719824 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.720689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.720967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.721236 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.721928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.723995 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.745582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1388/2780] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:42.749803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.760061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e85449-fece-4202-849b-ee719ca16d81 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.764823 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.765541 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.765753 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.766021 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.766742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.768571 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.779293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1393/2781] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:42.784623 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.794904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453484ff-fbe4-4385-828a-8c43e0d00b15 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.800584 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.801280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.801559 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.801832 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.802558 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.804199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e80be-9649-4164-8f57-50b772a9cbc6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.819577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1389/2782] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:07:42.820774 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:42.830831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e59d-c91c-41dc-acdf-a1e0ac17e227 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:42.835950 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:42.836844 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.837125 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.837397 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] PATH_INFO: `/v3/projects/680de1a369f141a080cd487bf77481a9/users/3eee44dc91e942ddaa05a9fad509fca3/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.853093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680de1a369f141a080cd487bf77481a9, user_id=3eee44dc91e942ddaa05a9fad509fca3, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:42.855727 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:42.870258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1394/2783] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:42 2022] PUT /identity/v3/projects/680de1a369f141a080cd487bf77481a9/users/3eee44dc91e942ddaa05a9fad509fca3/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 11:07:42.881434 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:42.881733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:42.882010 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:42.910599 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] MFA Rules not processed for user `3eee44dc91e942ddaa05a9fad509fca3`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:42.911838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:42.964327 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:42.987200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1390/2784] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:07:42.991800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.054683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5f2b6-8108-426a-9258-20fed0396bd9 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.059454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.060156 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.060452 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.060721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.061409 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.063301 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.090278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1395/2785] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:07:42 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:07:43.094443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.105833 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ed62-0f18-44e0-bd5e-030a8149c807 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.166636 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.167716 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.168039 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.168757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.169498 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.171497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.193106 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1391/2786] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:07:43 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:07:43.197311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.208205 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2738-6dff-400a-9963-8633c3ecc5ae admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.213578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.214268 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.214551 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.214831 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.215600 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.217267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.228490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1396/2787] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:43.232946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.243774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54163742-e6bc-40d7-bf62-2dbe7c20d65f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.249258 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.250190 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.250472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.250789 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] PATH_INFO: `/v3/projects/7d03d34e1658481d8acd863329f739cb/users/b84c6a20d05245d19a2bd735128bd4aa/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.267736 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d03d34e1658481d8acd863329f739cb, user_id=b84c6a20d05245d19a2bd735128bd4aa, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.270520 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.285754 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1392/2788] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:43 2022] PUT /identity/v3/projects/7d03d34e1658481d8acd863329f739cb/users/b84c6a20d05245d19a2bd735128bd4aa/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:43.289921 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.301771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d3c08-14f8-4c67-9a2c-e6ecc86ccfa0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.351956 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:43.362465 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.363345 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.363791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.364197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.365307 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.367107 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e421dfa-134a-45cc-84ed-372e6185275d admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.377300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1397/2789] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:07:43.381541 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.392664 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f8d9-1a74-43bc-8db2-b2d805bc69d8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.399830 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.400775 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.401058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.401332 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] PATH_INFO: `/v3/projects/7d03d34e1658481d8acd863329f739cb/users/b84c6a20d05245d19a2bd735128bd4aa/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.403454 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d03d34e1658481d8acd863329f739cb, user_id=b84c6a20d05245d19a2bd735128bd4aa, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.406150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.417812 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1393/2790] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:07:43 2022] PUT /identity/v3/projects/7d03d34e1658481d8acd863329f739cb/users/b84c6a20d05245d19a2bd735128bd4aa/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:07:43.428619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.428883 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.429161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.457800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] MFA Rules not processed for user `b84c6a20d05245d19a2bd735128bd4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:07:43.459022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:43.530181 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1398/2791] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:07:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:07:43.542218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:43.544658 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:07:43.630515 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a089b6-5bcb-4f4a-a64c-04cdfc2b3c20 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:43.635244 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:43.635907 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:43.636219 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:43.636472 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:43.675860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:43.713043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:43.722948 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:43.733054 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1394/2792] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:48.461720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:07:48.475399 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52257c9-953d-4ba7-b36b-fa142efc3eb3 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:07:48.479929 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:07:48.480646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:07:48.480931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:07:48.481207 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:07:48.491316 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:07:48.492779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26463d7b-ebff-4281-877c-7eb8e7296c5c service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:07:48.502584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1399/2793] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:07:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:07:53.729126 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=11 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:07:58.498295 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=45 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:10.502903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.571356 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79721548-0d65-4042-a90d-dc16ac9744ef service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.579808 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.581128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.581473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.581849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] PATH_INFO: `/v3/users/3eee44dc91e942ddaa05a9fad509fca3` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.583845 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eee44dc91e942ddaa05a9fad509fca3)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.586471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.641052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eee44dc91e942ddaa05a9fad509fca3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.655681 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eee44dc91e942ddaa05a9fad509fca3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.656411 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eee44dc91e942ddaa05a9fad509fca3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.663917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eee44dc91e942ddaa05a9fad509fca3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.673955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eee44dc91e942ddaa05a9fad509fca3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:10.680047 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1395/2794] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:10 2022] DELETE /identity/v3/users/3eee44dc91e942ddaa05a9fad509fca3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:10.684726 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.685313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.685313 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.704281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:10.705466 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.770780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1400/2795] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:10.780998 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:10.782479 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:10.842168 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7135-91f6-4576-a31c-a2906ebf5be4 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:10.845986 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:10.846625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:10.846904 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:10.847176 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:10.856863 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:10.858305 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:10.867990 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1396/2796] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:11.150100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.151635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.164121 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ada4b2-0524-45f0-9082-88905d8ba48c None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.169924 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.170800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.171081 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.171358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] PATH_INFO: `/v3/projects/680de1a369f141a080cd487bf77481a9` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.172598 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=680de1a369f141a080cd487bf77481a9)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.174492 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.224944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680de1a369f141a080cd487bf77481a9', 'request_id': 'req-77185e91-d44e-4df9-9a20-2f2506b4ce69'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.230794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680de1a369f141a080cd487bf77481a9', 'request_id': 'req-77185e91-d44e-4df9-9a20-2f2506b4ce69'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.236980 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680de1a369f141a080cd487bf77481a9', 'request_id': 'req-77185e91-d44e-4df9-9a20-2f2506b4ce69'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.255060 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] The token cache is being invalidate because project 680de1a369f141a080cd487bf77481a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:11.255533 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.256646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.259742 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1401/2797] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/projects/680de1a369f141a080cd487bf77481a9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:11.264550 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.277810 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54a96a-ffaa-476f-9ffa-1a7e3043dd31 service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.297689 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:11.340722 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.341592 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.341868 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.342129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] PATH_INFO: `/v3/users/b84c6a20d05245d19a2bd735128bd4aa` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.343370 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84c6a20d05245d19a2bd735128bd4aa)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.345357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.400030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84c6a20d05245d19a2bd735128bd4aa'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.413068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84c6a20d05245d19a2bd735128bd4aa'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.414090 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84c6a20d05245d19a2bd735128bd4aa'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.421395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84c6a20d05245d19a2bd735128bd4aa'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.430210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-fbaaa584-8214-4c21-b778-db01c69128a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84c6a20d05245d19a2bd735128bd4aa'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.436286 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1397/2798] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/users/b84c6a20d05245d19a2bd735128bd4aa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:11.716245 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:11.717779 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.733476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77185e91-d44e-4df9-9a20-2f2506b4ce69 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:11.791032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:11.791799 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.792109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.792379 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] PATH_INFO: `/v3/projects/7d03d34e1658481d8acd863329f739cb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.793546 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d03d34e1658481d8acd863329f739cb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:11.795775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:11.827421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:11.844604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d03d34e1658481d8acd863329f739cb', 'request_id': 'req-f4a4f495-a723-41f2-adae-264838d5fe35'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.849865 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d03d34e1658481d8acd863329f739cb', 'request_id': 'req-f4a4f495-a723-41f2-adae-264838d5fe35'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.856228 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d03d34e1658481d8acd863329f739cb', 'request_id': 'req-f4a4f495-a723-41f2-adae-264838d5fe35'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.870421 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] The token cache is being invalidate because project 7d03d34e1658481d8acd863329f739cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:11.870723 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.871328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-f4a4f495-a723-41f2-adae-264838d5fe35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:11.872900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1402/2799] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:11 2022] DELETE /identity/v3/projects/7d03d34e1658481d8acd863329f739cb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:11.877897 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.878376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.878490 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.896325 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:11.897597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:11.964131 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1398/2800] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:11.972427 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:11.972716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:11.972994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:11.992490 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:11.993719 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.003263 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1403/2801] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:12.007715 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.018083 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a396e6-90f5-4b81-8c42-e9cd2a9d1283 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.023231 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.023972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.024192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.024494 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.025220 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:12.025528 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.027149 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.042849 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1399/2802] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:12.048230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.059920 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab6db7-8bbb-4c70-b5e2-44877220350b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.065062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.065683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.065986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.066254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.066957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.068886 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.096989 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1404/2803] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:12.102707 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.113517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a208ba-6e6f-414d-bf1f-9090c7af1e88 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.139908 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.174605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.175270 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.175554 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.175829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.176549 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.178513 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.202751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1400/2804] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:12.207172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.218728 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2df47-9bd6-4070-9d1b-be0b1f62dbff admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.223526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.224224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.224505 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.224784 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.225515 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.227716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.238247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1405/2805] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.242175 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.252989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4f36-dbd4-4d6a-998b-8b29c93e99f6 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.258767 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.259417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.259692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.259996 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.260746 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.262379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c3396-9d7c-482e-8dfc-12cf36ffb075 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.273230 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1401/2806] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:12.277470 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.287596 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627d0bd-cc4d-490a-9565-7ec37e678589 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.293124 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.293933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.294244 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.294519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] PATH_INFO: `/v3/projects/be27498fd85340b4a72d06812efbe45b/users/ac52545c5e394077b18f7400433f296c/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.310254 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be27498fd85340b4a72d06812efbe45b, user_id=ac52545c5e394077b18f7400433f296c, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.312975 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.329168 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1406/2807] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:12 2022] PUT /identity/v3/projects/be27498fd85340b4a72d06812efbe45b/users/ac52545c5e394077b18f7400433f296c/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:12.339607 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.339910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.340206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.367523 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] MFA Rules not processed for user `ac52545c5e394077b18f7400433f296c`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.368826 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.440568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1402/2808] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:12.452733 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.454480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.515036 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7e5e8-e0d8-4095-9372-c6b9315eb417 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.519626 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.520300 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.520584 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.520863 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.530447 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.531913 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a3273-8b60-4ef9-a572-aa07bce596d5 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.541455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1407/2809] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:12.561387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.628300 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff909a19-d0e1-4253-bd1b-767f37fd3c9f None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.633978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.634848 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.635134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.635404 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] PATH_INFO: `/v3/users/ac52545c5e394077b18f7400433f296c` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.636699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac52545c5e394077b18f7400433f296c)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.638683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.684967 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.691147 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac52545c5e394077b18f7400433f296c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.703927 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac52545c5e394077b18f7400433f296c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.704608 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac52545c5e394077b18f7400433f296c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.711531 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac52545c5e394077b18f7400433f296c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.719467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac52545c5e394077b18f7400433f296c'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:12.725910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1403/2810] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:12 2022] DELETE /identity/v3/users/ac52545c5e394077b18f7400433f296c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:12.730464 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.730954 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.731065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.748423 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:12.749622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.798955 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:12.813720 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1408/2811] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:12.824103 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:12.825774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:12.885016 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e59a63-d7d4-4c15-8537-927e07948285 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:12.889174 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:12.889822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:12.890100 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:12.890368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:12.899748 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:12.901212 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17e676-668b-47b7-9ded-56dab919929d service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:12.910507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1404/2812] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:13.185342 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.186904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.200987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3200bde-a822-410b-a07d-fd76a9c04549 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.206491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.207442 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.207731 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.208042 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] PATH_INFO: `/v3/projects/be27498fd85340b4a72d06812efbe45b` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.209491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=be27498fd85340b4a72d06812efbe45b)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.211593 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.267259 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be27498fd85340b4a72d06812efbe45b', 'request_id': 'req-41b27b9a-6179-45c2-bc52-0f95819071eb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:13.275640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be27498fd85340b4a72d06812efbe45b', 'request_id': 'req-41b27b9a-6179-45c2-bc52-0f95819071eb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:13.284620 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be27498fd85340b4a72d06812efbe45b', 'request_id': 'req-41b27b9a-6179-45c2-bc52-0f95819071eb'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:13.304429 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] The token cache is being invalidate because project be27498fd85340b4a72d06812efbe45b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:13.304803 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:13.305791 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-41b27b9a-6179-45c2-bc52-0f95819071eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:13.308327 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1409/2813] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:13 2022] DELETE /identity/v3/projects/be27498fd85340b4a72d06812efbe45b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:13.316188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.316576 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.316988 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.336199 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:13.337395 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.399417 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1405/2814] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 11:08:13.410696 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.411111 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.411386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.432197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:13.433401 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.442691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1410/2815] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:13.447214 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.457560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8dd1-3daa-4dbe-aba6-bb2f7b59124d None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.462643 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.463294 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.463568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.463817 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.464611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:13.464917 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.466525 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.478289 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1406/2816] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:13.483007 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.492991 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b05a71-4621-4f94-ab3d-b44edb9e0377 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.498797 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.499448 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.499748 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.500056 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.500743 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.502635 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.530292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1411/2817] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:13.534923 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.551142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c065ae-c67e-4451-a8fe-6d501b770c93 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.609329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:13.625517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.626188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.626468 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.626740 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.627444 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.629461 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.654338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1407/2818] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:13.660587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.677543 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa85fe-2fac-4cbe-9098-940f424e2d20 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.684302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.684976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.685255 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.685526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.686250 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.687854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.699686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1412/2819] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:13.703574 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.714730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac39ac-4cbb-4e98-ad43-990db27db106 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.727283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.728074 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.728360 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.728634 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.729372 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.731625 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad2e20-5b6a-4820-8da7-0eb83c9a2630 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.746031 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1408/2820] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:13.750800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.762436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1c11-a1f9-4a31-96ec-5ae62c562a74 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:13.768455 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:13.769354 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.769632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.769906 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] PATH_INFO: `/v3/projects/ca152a75f2a348a189d6805eab00ee12/users/7e455d7c56434f08ba95bf049e5ee23f/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.787572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca152a75f2a348a189d6805eab00ee12, user_id=7e455d7c56434f08ba95bf049e5ee23f, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:13.790910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:13.806582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1413/2821] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:13 2022] PUT /identity/v3/projects/ca152a75f2a348a189d6805eab00ee12/users/7e455d7c56434f08ba95bf049e5ee23f/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:13.874651 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:13.874979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:13.875394 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:13.907334 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] MFA Rules not processed for user `7e455d7c56434f08ba95bf049e5ee23f`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:13.908566 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:13.978329 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1409/2822] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:13.990011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:13.992180 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.053074 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8d2ba-9c28-4b7a-ad04-236cab73af6b admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.056827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.057462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.057745 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.058015 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.066281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.067706 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd7889-062e-4c21-829e-6b6e106f1099 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.077065 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1414/2823] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:14.099751 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.177540 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba491f-f7c2-4cdc-affa-7738f368b2c1 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.184532 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.185416 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.185690 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.185961 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] PATH_INFO: `/v3/users/7e455d7c56434f08ba95bf049e5ee23f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.187341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e455d7c56434f08ba95bf049e5ee23f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.189341 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.242339 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e455d7c56434f08ba95bf049e5ee23f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.251514 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:14.259028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e455d7c56434f08ba95bf049e5ee23f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.260115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e455d7c56434f08ba95bf049e5ee23f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.268141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e455d7c56434f08ba95bf049e5ee23f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.276503 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e455d7c56434f08ba95bf049e5ee23f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.282934 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1410/2824] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:14 2022] DELETE /identity/v3/users/7e455d7c56434f08ba95bf049e5ee23f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:14.287766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.288119 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.288428 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.305986 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:14.307174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.317100 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:14.369530 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1415/2825] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:14.379627 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.381350 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.446227 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd73d-e895-4833-98b5-ee9c052b5044 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.450976 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.451771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.452094 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.452376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.462699 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.464192 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cf15-f222-4642-8aa8-73bc2eba4809 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.474140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1411/2826] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:14.757961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:14.759491 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.771508 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9ab89-c8c9-4d0d-8198-5b46b63942f4 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:14.776915 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:14.777849 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.778131 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.778526 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] PATH_INFO: `/v3/projects/ca152a75f2a348a189d6805eab00ee12` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.780305 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca152a75f2a348a189d6805eab00ee12)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:14.782527 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:14.832703 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca152a75f2a348a189d6805eab00ee12', 'request_id': 'req-51ade77c-af3e-4b71-ad6f-72c804a19843'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.838688 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca152a75f2a348a189d6805eab00ee12', 'request_id': 'req-51ade77c-af3e-4b71-ad6f-72c804a19843'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.845288 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca152a75f2a348a189d6805eab00ee12', 'request_id': 'req-51ade77c-af3e-4b71-ad6f-72c804a19843'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.862022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] The token cache is being invalidate because project ca152a75f2a348a189d6805eab00ee12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:14.862321 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.863079 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-51ade77c-af3e-4b71-ad6f-72c804a19843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:14.864818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1416/2827] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:14 2022] DELETE /identity/v3/projects/ca152a75f2a348a189d6805eab00ee12 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:14.870162 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.870452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.870728 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.889571 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:14.890782 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.957381 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1412/2828] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:14.965513 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:14.965798 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:14.966073 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:14.985510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:14.986717 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:14.996324 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1417/2829] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:15.000796 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.011183 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f9a2-f383-498b-a52d-ef9d0cff7df2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.017021 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.017683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.017965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.018297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.019018 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:15.019323 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.020978 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.032926 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1413/2830] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:15.037186 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.047524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dfd66-8bb6-4112-b461-8e4f6a9d2b61 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.053137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.053794 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.054240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.054557 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.055203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.057140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.085434 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1418/2831] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:15 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:15.089815 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.100213 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1795-f5de-4298-b928-48eb4e7e63f9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.157275 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.157946 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.158228 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.158507 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.159188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.161141 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.170265 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:15.186944 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1414/2832] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:15 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:15.191999 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.204406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accb25f-eb34-4c0e-892f-15a2f8b1655c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.209964 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.211093 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.211380 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.211659 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.212424 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.214032 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.224914 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1419/2833] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:15.229473 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.239855 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b53bb-432b-46e8-91f3-b7b2a6ee7f9e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.248283 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.249255 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.249593 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.249992 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.251127 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.253654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056134-4396-417d-b4f1-5c7e79d408f1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.265396 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1415/2834] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:15.269494 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.280113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11dc5c-8b42-4dd2-97bb-246537a6f16f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.285016 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.285911 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.286220 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.286495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] PATH_INFO: `/v3/projects/d337d17b54da452a893c760564e41ddb/users/264406dc034b4e39a2e7341bb4f2b14f/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.302366 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d337d17b54da452a893c760564e41ddb, user_id=264406dc034b4e39a2e7341bb4f2b14f, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.305291 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.322587 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1420/2835] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:15 2022] PUT /identity/v3/projects/d337d17b54da452a893c760564e41ddb/users/264406dc034b4e39a2e7341bb4f2b14f/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:15.334226 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.334553 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.334835 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.361266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] MFA Rules not processed for user `264406dc034b4e39a2e7341bb4f2b14f`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:15.362469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:15.427942 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1416/2836] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:15.458862 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.461206 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:15.514161 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:15.541940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664615ec-6521-4b8f-827d-9f21a6cdd07c admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.546367 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.547045 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.547322 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.547754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.557780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.559961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33e1e-6daf-477a-b0b9-679bc1de25f2 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.572691 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1421/2837] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:15.587005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.654193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6c87e-746f-476f-8cfe-3f38172fdfa2 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.660034 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.660909 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.661189 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.661502 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] PATH_INFO: `/v3/users/264406dc034b4e39a2e7341bb4f2b14f` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.662750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264406dc034b4e39a2e7341bb4f2b14f)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.664890 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.720166 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264406dc034b4e39a2e7341bb4f2b14f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.733020 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264406dc034b4e39a2e7341bb4f2b14f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.733771 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264406dc034b4e39a2e7341bb4f2b14f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.742081 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264406dc034b4e39a2e7341bb4f2b14f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.743757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:15.751683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264406dc034b4e39a2e7341bb4f2b14f'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:15.758530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1417/2838] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:15 2022] DELETE /identity/v3/users/264406dc034b4e39a2e7341bb4f2b14f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:15.763581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.764112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.764224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.786368 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:15.787582 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:15.859403 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1422/2839] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:15.869800 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:15.871504 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:15.931082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9592a-fd55-4e26-8c1b-f032ce8c6c29 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:15.935052 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:15.935694 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:15.936005 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:15.936279 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:15.945853 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:15.947302 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aed4ef-ef5c-480c-b301-dcd4b5aa9e8c service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:15.956825 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1418/2840] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:16.249572 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.251025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.265483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4edc34-739a-42f7-b006-001d73ed7dcc None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.271744 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.273153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.273553 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.273910 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] PATH_INFO: `/v3/projects/d337d17b54da452a893c760564e41ddb` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.275290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d337d17b54da452a893c760564e41ddb)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.277213 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.325524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd337d17b54da452a893c760564e41ddb', 'request_id': 'req-1c837a37-9e1b-44f9-823b-1f346db15719'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.331097 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd337d17b54da452a893c760564e41ddb', 'request_id': 'req-1c837a37-9e1b-44f9-823b-1f346db15719'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.337386 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd337d17b54da452a893c760564e41ddb', 'request_id': 'req-1c837a37-9e1b-44f9-823b-1f346db15719'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.355683 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] The token cache is being invalidate because project d337d17b54da452a893c760564e41ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:16.355990 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.356621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-1c837a37-9e1b-44f9-823b-1f346db15719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:16.358197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1423/2841] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:16 2022] DELETE /identity/v3/projects/d337d17b54da452a893c760564e41ddb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:16.363115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.363597 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.363713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.381254 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.382478 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.442721 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1419/2842] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:16.451280 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.451646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.452054 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.473604 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.474806 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.483909 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1424/2843] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:16.488215 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.498133 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561268d9-017c-4d2a-99b4-341a337f3c3a None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.503117 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.503761 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.504062 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.504358 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.505075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:16.505399 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.507066 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.518292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1420/2844] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:16.522786 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.531775 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5aeb7-b31a-4db6-a001-d0c6be09a571 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.536859 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.537524 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.537811 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.538082 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.538777 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.540710 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.568495 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1425/2845] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:16.572604 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.582268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03914e6-7581-47b7-9bd0-3ae652c76983 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.638899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.639552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.639829 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.640134 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.640818 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.642777 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.670457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1421/2846] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 11:08:16.674632 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.686467 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f743202-e11c-46eb-8cbc-75bd6c2ef9fd admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.692812 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.693476 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.693764 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.694061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.694834 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.696480 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.707309 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1426/2847] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:16.711676 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.722425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440f952-ee42-4e67-8c5d-bd8ac781221f admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.728458 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.729109 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.729364 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.729655 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.730376 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.731991 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3f81b-916d-45f3-b1f7-ece1910356a8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.737389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:16.744129 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1422/2848] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:16.747895 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.758362 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15129cf-e27a-42a1-9863-a2dd2b1701cc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.764149 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.765025 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.765302 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.765577 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] PATH_INFO: `/v3/projects/3a599d5507e442fcba0540322df84b60/users/6949d30aef01485ca74d9e68aa7f8398/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.780296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a599d5507e442fcba0540322df84b60, user_id=6949d30aef01485ca74d9e68aa7f8398, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:16.782918 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:16.792190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:16.798902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1427/2849] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:16 2022] PUT /identity/v3/projects/3a599d5507e442fcba0540322df84b60/users/6949d30aef01485ca74d9e68aa7f8398/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:16.809457 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.809742 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.810028 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.839260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] MFA Rules not processed for user `6949d30aef01485ca74d9e68aa7f8398`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:16.840530 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.912347 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1423/2850] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:16.924406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:16.926373 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:16.984941 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cb3e0-4c0f-46d4-bfba-eca7d2f8cfc8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:16.989229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:16.989884 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:16.990164 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:16.990436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:16.999317 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:17.000780 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43a0ac-04d7-4aef-84b5-a2d83e8aa43d service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:17.009878 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1428/2851] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:26.907318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=32 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:27.005854 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=27 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:34.746737 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:34.812466 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b295e0-d727-419b-8f7a-a511694b4492 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:34.817857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:34.818750 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:34.819027 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:34.819298 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] PATH_INFO: `/v3/users/6949d30aef01485ca74d9e68aa7f8398` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:34.820567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6949d30aef01485ca74d9e68aa7f8398)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:34.822497 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:34.870903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6949d30aef01485ca74d9e68aa7f8398'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:34.882766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6949d30aef01485ca74d9e68aa7f8398'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:34.883465 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6949d30aef01485ca74d9e68aa7f8398'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:34.890422 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6949d30aef01485ca74d9e68aa7f8398'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:34.899568 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6949d30aef01485ca74d9e68aa7f8398'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:34.905626 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1424/2852] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:34 2022] DELETE /identity/v3/users/6949d30aef01485ca74d9e68aa7f8398 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:34.910454 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:34.910945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:34.910945 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:34.929038 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:34.930293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:34.990554 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1429/2853] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:35.002297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.004776 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:35.068354 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d022f-3234-412a-a739-708025704e2b admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.073343 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.074024 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.074304 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.074589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.084280 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.085791 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c940dc2-f8f9-403a-88ac-38c465a8d7a3 service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.095451 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1425/2854] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:35.382578 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.384267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:35.396414 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da246e9a-8ba3-45be-b6fd-cea7e280541b None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.401875 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.402782 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.403061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.403330 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] PATH_INFO: `/v3/projects/3a599d5507e442fcba0540322df84b60` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.404810 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a599d5507e442fcba0540322df84b60)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.406767 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.457337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a599d5507e442fcba0540322df84b60', 'request_id': 'req-573ca768-e9f9-40b9-99e9-a31f4863c189'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:35.462931 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a599d5507e442fcba0540322df84b60', 'request_id': 'req-573ca768-e9f9-40b9-99e9-a31f4863c189'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:35.469203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a599d5507e442fcba0540322df84b60', 'request_id': 'req-573ca768-e9f9-40b9-99e9-a31f4863c189'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:35.483666 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] The token cache is being invalidate because project 3a599d5507e442fcba0540322df84b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95327) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:35.483997 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:35.484654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-573ca768-e9f9-40b9-99e9-a31f4863c189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:35.486232 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1430/2855] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:35 2022] DELETE /identity/v3/projects/3a599d5507e442fcba0540322df84b60 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:35.491266 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.491560 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.491950 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.510379 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:35.511605 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:35.528903 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:35.578606 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1426/2856] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:35.587621 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.587936 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.588215 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.607483 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:35.608711 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:35.618003 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1431/2857] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:35.623030 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.636816 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2566b43-9438-455e-b025-cc275012fb22 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.642516 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.643172 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.643452 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.643724 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] PATH_INFO: `/v3/domains` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.644487 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=95328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 11:08:35.644797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.646441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.658977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1427/2858] 10.222.0.27 () {58 vars in 1152 bytes} [Thu Jul 14 11:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 11:08:35.663398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.673562 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135709-8db8-4240-ba70-f8f6cba640c7 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.679440 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.680129 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.680407 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.680674 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.681358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.683260 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.711904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1432/2859] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:35 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:35.717891 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.729019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1df4-8933-44b6-b000-6db4ca04c35e admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.789037 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.789686 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.789977 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.790251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.790955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.792919 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.821075 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1428/2860] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:35 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:35.825339 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.835827 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cf2d7-f357-4e03-bc52-c5d38556ef44 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.841585 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.842369 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.842654 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.842925 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.843656 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.845285 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.855902 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1433/2861] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:35.859937 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.870493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae605f86-b56e-40c7-91da-562cf147d367 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.875705 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.876425 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.876713 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.876987 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.877726 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.879384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4793c9-0a31-4b17-998a-e3f5809b70b0 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.889753 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1429/2862] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 11:08:35.893800 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:35.904047 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5415-2224-4a5f-9af7-a8321ba26d3e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:35.909099 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:35.909987 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.910264 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.910535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] PATH_INFO: `/v3/projects/68fab1380e824b35846d5770a955e163/users/5c435ece2cac47e78e71fb0d4729351e/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.926258 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fab1380e824b35846d5770a955e163, user_id=5c435ece2cac47e78e71fb0d4729351e, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:35.928979 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:35.943927 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1434/2863] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:35 2022] PUT /identity/v3/projects/68fab1380e824b35846d5770a955e163/users/5c435ece2cac47e78e71fb0d4729351e/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:35.954277 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:35.954564 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:35.954838 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:35.983284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] MFA Rules not processed for user `5c435ece2cac47e78e71fb0d4729351e`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:35.984730 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:36.051563 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1430/2864] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:36.056532 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.077586 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:36.118933 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60acb7e-1f83-45fb-8be7-037f2edbefdc admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.123730 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.124436 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.124716 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.124995 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.125661 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.127558 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.154493 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1435/2865] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:36.158743 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.171292 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a0124-d58e-4f02-9117-90dc48b80379 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.184482 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:36.231999 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.232711 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.232938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.233211 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.233887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.235822 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.261561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1431/2866] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:36.265783 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.276496 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e0b7-dcd5-4a54-8ea9-41330fa11baa admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.281358 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.282012 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.282293 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.282564 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.283292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.284926 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.296137 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1436/2867] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:36.300180 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.310770 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff820ce-5ed2-4b0b-9423-524416e72812 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.315670 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.316357 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.316672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.316922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] PATH_INFO: `/v3/roles` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.317661 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.319245 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40babfb0-874a-42d6-8cab-e5c66d59c020 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.329345 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1432/2868] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:36.333328 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.343643 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d15a69-ee5d-4e2d-992c-e3c783ba9522 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.348824 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.349692 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.349971 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.350240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] PATH_INFO: `/v3/projects/b09469960cfd4efcb742b8b631e33d30/users/4efa906b42434cea8a3af923eb4a9ffd/roles/8cde109646d74ea789591d2724109ef7` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.365978 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09469960cfd4efcb742b8b631e33d30, user_id=4efa906b42434cea8a3af923eb4a9ffd, role_id=8cde109646d74ea789591d2724109ef7)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.368819 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.385770 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1437/2869] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:36 2022] PUT /identity/v3/projects/b09469960cfd4efcb742b8b631e33d30/users/4efa906b42434cea8a3af923eb4a9ffd/roles/8cde109646d74ea789591d2724109ef7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:36.403491 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.403842 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.404297 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.436257 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] MFA Rules not processed for user `4efa906b42434cea8a3af923eb4a9ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:36.437483 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:36.502431 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1433/2870] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:36.506974 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.571173 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596b555-aaf9-45db-904c-8a675780c1f8 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.576700 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.577357 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.577640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.577916 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] PATH_INFO: `/v3/projects` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.578603 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.580560 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.611597 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1438/2871] 10.222.0.27 () {60 vars in 1152 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:36.615807 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.627580 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624fe66-3fda-4d39-b4e7-531ef580f3ba None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.687058 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.687733 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.688055 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.688336 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] PATH_INFO: `/v3/users` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.689043 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.690981 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.712582 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1434/2872] 10.222.0.27 () {60 vars in 1143 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 11:08:36.716788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.727176 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff407-d25e-4e57-9403-02a74c78250f admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.732011 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.732672 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.732957 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.733267 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.733993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.735592 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.746355 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1439/2873] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:36.751121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.767269 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bd92d-a464-49e3-9ac6-39ce7d8a02b3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.774508 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.775391 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.775675 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.775980 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] PATH_INFO: `/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb/users/16c92db431624d45bfb8efd9b206226c/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.788004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:36.792221 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb, user_id=16c92db431624d45bfb8efd9b206226c, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.795061 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.809912 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1435/2874] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:36 2022] PUT /identity/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb/users/16c92db431624d45bfb8efd9b206226c/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:36.814022 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.828970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4361e9-3650-49e3-840f-430d4ec749a1 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.874970 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:36.891504 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.892230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.892517 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.892795 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] PATH_INFO: `/v3/roles` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.893528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.895144 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba308-804e-421e-a778-a4360ac3b007 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.905241 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1440/2875] 10.222.0.27 () {58 vars in 1121 bytes} [Thu Jul 14 11:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 11:08:36.909602 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:36.921091 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2762-8466-4e40-902d-ab945546d1fa admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:36.926682 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:36.927578 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.927854 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.928161 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] PATH_INFO: `/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb/users/16c92db431624d45bfb8efd9b206226c/roles/bd93d91744f54f739650facbdec5aef5` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.930384 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb, user_id=16c92db431624d45bfb8efd9b206226c, role_id=bd93d91744f54f739650facbdec5aef5)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:36.933084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:36.944102 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1436/2876] 10.222.0.27 () {60 vars in 1482 bytes} [Thu Jul 14 11:08:36 2022] PUT /identity/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb/users/16c92db431624d45bfb8efd9b206226c/roles/bd93d91744f54f739650facbdec5aef5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:36.954619 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33307-6286-405d-8692-e86358c9592f None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:36.955128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33307-6286-405d-8692-e86358c9592f None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:36.955128 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee33307-6286-405d-8692-e86358c9592f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:36.983040 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-bee33307-6286-405d-8692-e86358c9592f None None] MFA Rules not processed for user `16c92db431624d45bfb8efd9b206226c`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:36.984311 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bee33307-6286-405d-8692-e86358c9592f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:37.062904 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1441/2877] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:37.074938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:37.076938 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:37.138150 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7ef4-457d-49ad-9907-ad8099f407b9 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:37.143011 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:37.143657 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:37.144035 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:37.144315 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:37.202972 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:37.204469 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:37.213624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1437/2878] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:38.728169 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33307-6286-405d-8692-e86358c9592f None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:38.729788 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-bee33307-6286-405d-8692-e86358c9592f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:38.749061 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee33307-6286-405d-8692-e86358c9592f None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:38.753627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:38.754278 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:38.754561 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:38.754833 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:38.816871 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:38.818153 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:38.828225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1442/2879] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:39.383426 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:39.392488 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0fdab-15d8-405e-87ab-1d3ee3b206fe service cinder] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:39.397071 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13be832163b9406583f63f7cd00651ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:39.397709 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:39.397993 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:39.398260 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:39.408641 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:39.410084 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83529058-8bb7-4076-a98d-ba5788a04912 service cinder] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:39.420338 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1438/2880] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:43.491143 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:43.557462 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0d7e2-1483-4ad0-8f38-66f0401c41d8 service cinder] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:43.562762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:43.563646 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.563944 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.564230 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] PATH_INFO: `/v3/users/5c435ece2cac47e78e71fb0d4729351e` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.565510 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c435ece2cac47e78e71fb0d4729351e)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:43.567474 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:43.573963 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=100 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:43.619570 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c435ece2cac47e78e71fb0d4729351e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:43.631398 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c435ece2cac47e78e71fb0d4729351e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:43.632117 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c435ece2cac47e78e71fb0d4729351e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:43.638976 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c435ece2cac47e78e71fb0d4729351e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:43.648027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c435ece2cac47e78e71fb0d4729351e'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:43.654347 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1443/2881] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:43 2022] DELETE /identity/v3/users/5c435ece2cac47e78e71fb0d4729351e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:43.658922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] REQUEST_METHOD: `POST` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.659443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.659443 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.678552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.auth.core [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] MFA Rules not processed for user `e56c84bcaa9a49c8bd5f12e10aeab889`. Rule list: `[]` (Enabled: `True`). {{(pid=95328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:08:43.679766 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:43.747140 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1439/2882] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:08:43.754109 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:43.755610 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:43.814284 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc601-8f11-4dce-98be-4905cf125c46 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:43.818813 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:43.819468 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:43.819754 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:43.820071 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:43.829639 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:43.831110 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:43.840959 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1444/2883] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:08:44.117284 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:44.119206 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:44.133989 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ca21-8375-4514-bb6a-187739f08594 None None] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:44.140328 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:44.141121 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:44.141441 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:44.141642 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] PATH_INFO: `/v3/projects/68fab1380e824b35846d5770a955e163` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:44.142965 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fab1380e824b35846d5770a955e163)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:44.144928 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:44.162125 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:44.194936 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fab1380e824b35846d5770a955e163', 'request_id': 'req-e572a6b1-af22-4da8-83ee-4d15c76217b3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.201004 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fab1380e824b35846d5770a955e163', 'request_id': 'req-e572a6b1-af22-4da8-83ee-4d15c76217b3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.207068 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fab1380e824b35846d5770a955e163', 'request_id': 'req-e572a6b1-af22-4da8-83ee-4d15c76217b3'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.220561 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] The token cache is being invalidate because project 68fab1380e824b35846d5770a955e163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:44.220834 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.221534 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.223171 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1440/2884] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:44 2022] DELETE /identity/v3/projects/68fab1380e824b35846d5770a955e163 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:44.227292 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:44.237697 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b475bf-d14a-4a5a-9abb-ae7cde2944ad service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:44.298393 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:44.299190 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:44.299473 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:44.299747 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] PATH_INFO: `/v3/users/4efa906b42434cea8a3af923eb4a9ffd` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:44.301009 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4efa906b42434cea8a3af923eb4a9ffd)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:44.302969 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:44.334900 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:44.353686 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4efa906b42434cea8a3af923eb4a9ffd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.365528 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4efa906b42434cea8a3af923eb4a9ffd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.366182 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4efa906b42434cea8a3af923eb4a9ffd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.372940 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4efa906b42434cea8a3af923eb4a9ffd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.381262 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4efa906b42434cea8a3af923eb4a9ffd'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.387523 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1445/2885] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:44 2022] DELETE /identity/v3/users/4efa906b42434cea8a3af923eb4a9ffd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:44.673704 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:44.675218 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:44.689678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a6b1-af22-4da8-83ee-4d15c76217b3 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:44.749342 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:44.750210 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:44.750493 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:44.750772 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] PATH_INFO: `/v3/projects/b09469960cfd4efcb742b8b631e33d30` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:44.752017 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09469960cfd4efcb742b8b631e33d30)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:44.753955 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:44.803407 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09469960cfd4efcb742b8b631e33d30', 'request_id': 'req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.812242 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09469960cfd4efcb742b8b631e33d30', 'request_id': 'req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.819710 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09469960cfd4efcb742b8b631e33d30', 'request_id': 'req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.834899 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] The token cache is being invalidate because project b09469960cfd4efcb742b8b631e33d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:44.835188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.835860 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.837959 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1441/2886] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:44 2022] DELETE /identity/v3/projects/b09469960cfd4efcb742b8b631e33d30 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:44.842172 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:44.843756 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:44.857463 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d766e0-2944-437d-9096-edbc1a80d9b0 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:44.916127 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:44.917013 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:44.917337 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:44.917617 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] PATH_INFO: `/v3/users/16c92db431624d45bfb8efd9b206226c` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:44.919031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c92db431624d45bfb8efd9b206226c)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:44.921477 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:44.978174 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c92db431624d45bfb8efd9b206226c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.992140 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c92db431624d45bfb8efd9b206226c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:44.993203 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c92db431624d45bfb8efd9b206226c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.000625 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c92db431624d45bfb8efd9b206226c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.010627 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-e3772301-e78d-4e17-9f40-bc35cdb62d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c92db431624d45bfb8efd9b206226c'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.017116 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1446/2887] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:08:44 2022] DELETE /identity/v3/users/16c92db431624d45bfb8efd9b206226c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:45.309082 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:08:45.310536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:08:45.323827 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26eb26-63d5-4bdc-b95e-e6fa779290b1 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:08:45.355250 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:45.383321 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:08:45.384184 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:08:45.384467 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:08:45.384744 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] PATH_INFO: `/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:08:45.385957 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:08:45.387902 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:08:45.437844 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d7e1ed2c8c4f6a8a8f330ff3c9c8eb', 'request_id': 'req-b90e0a76-70f1-4d8b-ac61-609d070c5f52'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.444115 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d7e1ed2c8c4f6a8a8f330ff3c9c8eb', 'request_id': 'req-b90e0a76-70f1-4d8b-ac61-609d070c5f52'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.450268 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d7e1ed2c8c4f6a8a8f330ff3c9c8eb', 'request_id': 'req-b90e0a76-70f1-4d8b-ac61-609d070c5f52'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.466421 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] The token cache is being invalidate because project d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:08:45.466692 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.467318 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:08:45.468915 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1442/2888] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:08:45 2022] DELETE /identity/v3/projects/d1d7e1ed2c8c4f6a8a8f330ff3c9c8eb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:08:55.013569 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:08:55.463696 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:28.535177 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] REQUEST_METHOD: `POST` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:28.535746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:28.535746 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:28.555418 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.auth.core [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] MFA Rules not processed for user `2e2b4b08c1b34a7f86996a806953ac1d`. Rule list: `[]` (Enabled: `True`). {{(pid=95327) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 11:09:28.557112 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:28.614405 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1447/2889] 10.222.0.27 () {58 vars in 957 bytes} [Thu Jul 14 11:09:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 11:09:28.963181 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:28.964925 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:29.026040 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e0a76-70f1-4d8b-ac61-609d070c5f52 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:29.030264 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:29.030910 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:29.031194 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:29.031471 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:29.033375 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:29.043142 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:29.044611 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:29.053193 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1443/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:31.843417 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:31.845085 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.941640 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cbacd-7856-48a8-b87f-d290b635fe38 None None] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:31.948196 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:31.948581 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:31.948872 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:31.949296 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:31.951132 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:31.965996 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:31.967818 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:31.981075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1448/2891] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:39.049552 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=26 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:41.359065 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:09:41.361613 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:41.382894 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd44b80-211d-4d51-afde-30694ead80ad service neutron] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:09:41.391542 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:09:41.392539 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:09:41.392856 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:09:41.393136 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:09:41.396153 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:09:41.414449 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:09:41.416099 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:09:41.426377 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1444/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:09:41.975633 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=59 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:09:51.421481 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=6 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:10:40.863689 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:10:40.865269 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:40.941607 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5ed0-eb27-462e-9770-826d83856c06 service nova] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:10:40.947005 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:10:40.947708 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:10:40.948089 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:10:40.948364 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:10:40.950290 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:10:40.962840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:10:40.965208 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:10:40.975563 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1449/2893] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:10:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:10:50.971202 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=32 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:21.021589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:21.033500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb937-d88d-4a10-9db6-382ff6133824 service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:21.040509 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08cdb2d283143aa83b38bb3b04aee8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:21.041307 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:21.041589 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:21.041864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:21.043979 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:21.058635 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:21.060412 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:21.073484 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1445/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:27.016838 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:27.027031 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39bbc7-c7ab-4175-9496-00d15ef6cc09 service placement] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:27.030965 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a3d1b0e64049d28adc1a4f9ea10337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:27.031622 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:27.031961 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:27.032229 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:27.034240 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:27.106519 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:27.107994 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:27.117840 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1450/2895] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:30.149299 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:30.150922 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:30.170128 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a1406-b9e5-4cf7-b9b1-3c9730174437 service nova] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:30.175188 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:30.175857 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] REQUEST_METHOD: `GET` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:30.176287 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:30.176577 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:30.178654 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:30.193200 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:30.194918 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:30.207019 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1446/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:11:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:11:33.487981 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:33.490534 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:33.502702 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d6c0-5eef-4603-a2dc-8588c5c00c12 service neutron] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:33.507270 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:33.508511 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:33.508793 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:33.509062 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] PATH_INFO: `/v3/users/2e2b4b08c1b34a7f86996a806953ac1d` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:33.510261 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2b4b08c1b34a7f86996a806953ac1d)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:33.512224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:33.562002 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2b4b08c1b34a7f86996a806953ac1d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.573901 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2b4b08c1b34a7f86996a806953ac1d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.574535 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2b4b08c1b34a7f86996a806953ac1d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.581771 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2b4b08c1b34a7f86996a806953ac1d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.591281 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2b4b08c1b34a7f86996a806953ac1d'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.597762 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1451/2897] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:11:33 2022] DELETE /identity/v3/users/2e2b4b08c1b34a7f86996a806953ac1d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:11:33.786387 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:33.797797 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f65735-e441-4501-b08d-9e0356bf4141 service placement] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:33.864366 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:33.865259 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:33.865536 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:33.865837 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] PATH_INFO: `/v3/projects/fea9c5d098d646b89213d35f737e2308` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:33.867500 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea9c5d098d646b89213d35f737e2308)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:33.869543 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:33.926368 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea9c5d098d646b89213d35f737e2308', 'request_id': 'req-819bdca2-1462-4649-80b3-8adde78250f8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.933763 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea9c5d098d646b89213d35f737e2308', 'request_id': 'req-819bdca2-1462-4649-80b3-8adde78250f8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.939624 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea9c5d098d646b89213d35f737e2308', 'request_id': 'req-819bdca2-1462-4649-80b3-8adde78250f8'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.957235 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] The token cache is being invalidate because project fea9c5d098d646b89213d35f737e2308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:11:33.957581 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.958256 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:33.960521 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1447/2898] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:11:33 2022] DELETE /identity/v3/projects/fea9c5d098d646b89213d35f737e2308 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:11:33.964740 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:33.966225 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:33.980224 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816683a9-f3ef-4184-a16e-85f6afa70b95 admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:34.037514 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:34.038435 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:34.038721 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:34.038993 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] PATH_INFO: `/v3/users/d2a44e619a294756ae6ab8a105ac64d8` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:34.040319 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a44e619a294756ae6ab8a105ac64d8)` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:34.042247 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:34.048066 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:34.092197 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a44e619a294756ae6ab8a105ac64d8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.104095 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a44e619a294756ae6ab8a105ac64d8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.104766 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a44e619a294756ae6ab8a105ac64d8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.111406 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a44e619a294756ae6ab8a105ac64d8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.120184 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.notifications [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a44e619a294756ae6ab8a105ac64d8'} {{(pid=95327) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.126372 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1452/2899] 10.222.0.27 () {58 vars in 1223 bytes} [Thu Jul 14 11:11:33 2022] DELETE /identity/v3/users/d2a44e619a294756ae6ab8a105ac64d8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:11:34.306774 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Authenticating user token {{(pid=95328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:11:34.308397 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.fernet_utils [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:11:34.322672 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdca2-1462-4649-80b3-8adde78250f8 admin admin] Validating token access rules against request {{(pid=95328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:11:34.378757 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56c84bcaa9a49c8bd5f12e10aeab889', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd0314351a554271b0b9af50b68ac028', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:11:34.379678 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:11:34.379962 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:11:34.380251 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] PATH_INFO: `/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d` {{(pid=95328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:11:34.381517 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b994fb844e01431ea8e49c1a5d0ad61d)` {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:11:34.383567 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] RBAC: Authorization granted {{(pid=95328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:11:34.389887 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:34.434864 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b994fb844e01431ea8e49c1a5d0ad61d', 'request_id': 'req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.441014 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b994fb844e01431ea8e49c1a5d0ad61d', 'request_id': 'req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.446958 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b994fb844e01431ea8e49c1a5d0ad61d', 'request_id': 'req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2'} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.461389 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] The token cache is being invalidate because project b994fb844e01431ea8e49c1a5d0ad61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=95328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 11:11:34.461683 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.462380 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG keystone.notifications [None req-7062deeb-5e33-49ff-a5bb-08c3848fb5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=95328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 11:11:34.464013 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: [pid: 95328|app: 0|req: 1448/2900] 10.222.0.27 () {58 vars in 1232 bytes} [Thu Jul 14 11:11:34 2022] DELETE /identity/v3/projects/b994fb844e01431ea8e49c1a5d0ad61d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 11:11:44.123027 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:11:44.458344 dv-cinder-846146-7-iscsi devstack@keystone.service[95328]: DEBUG dbcounter [-] [95328] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=95328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 11:15:41.760170 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Authenticating user token {{(pid=95327) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 11:15:41.761877 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:41.914075 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491c2ae-0f0f-4cad-a210-9610585caa4e admin admin] Validating token access rules against request {{(pid=95327) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 11:15:41.985113 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17f4edebe2ed45a08ef7460d812c563d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b49330c5b3874cde8c864bc5b1a98a69', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=95327) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 11:15:41.985778 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] REQUEST_METHOD: `GET` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 11:15:41.986057 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] SCRIPT_NAME: `/identity` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 11:15:41.986332 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=95327) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 11:15:41.988375 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.fernet_utils [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=95327) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 11:15:42.000475 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 11:15:42.002069 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb1fd5-61ff-44c4-acb8-478ee9851c84 service placement] RBAC: Authorization granted {{(pid=95327) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 11:15:42.012861 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: [pid: 95327|app: 0|req: 1453/2901] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Jul 14 11:15:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 11:15:52.008664 dv-cinder-846146-7-iscsi devstack@keystone.service[95327]: DEBUG dbcounter [-] [95327] Writing DB stats keystone:SELECT=89 {{(pid=95327) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}